Categorisation of continuous covariates for stratified randomisation: How should we adjust?

To obtain valid inference following stratified randomisation, treatment effects should be estimated with adjustment for stratification variables. Stratification sometimes requires categorisation of a continuous prognostic variable (eg, age), which raises the question: should adjustment be based on randomisation categories or underlying continuous values? In practice, adjustment for randomisation categories is more common. We reviewed trials published in general medical journals and found none of the 32 trials that stratified randomisation based on a continuous variable adjusted for continuous values in the primary analysis. Using data simulation, this article evaluates the performance of different adjustment strategies for continuous and binary outcomes where the covariate-outcome relationship (via the link function) was either linear or non-linear. Given the utility of covariate adjustment for addressing missing data, we also considered settings with complete or missing outcome data. Analysis methods included linear or logistic regression with no adjustment for the stratification variable, adjustment for randomisation categories, or adjustment for continuous values assuming a linear covariate-outcome relationship or allowing for non-linearity using fractional polynomials or restricted cubic splines. Unadjusted analysis performed poorly throughout. Adjustment approaches that misspecified the underlying covariate-outcome relationship were less powerful and, alarmingly, biased in settings where the stratification variable predicted missing outcome data. Adjustment for randomisation categories tends to involve the highest degree of misspecification, and so should be avoided in practice. To guard against misspecification, we recommend use of flexible approaches such as fractional polynomials and restricted cubic splines when adjusting for continuous stratification variables in randomised trials.

Eliminating Ambiguous Treatment Effects Using Estimands.

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly.

Handling misclassified stratification variables in the analysis of randomised trials with continuous outcomes.

Many trials use stratified randomisation, where participants are randomised within strata defined by one or more baseline covariates. While it is important to adjust for stratification variables in the analysis, the appropriate method of adjustment is unclear when stratification variables are affected by misclassification and hence some participants are randomised in the incorrect stratum. We conducted a simulation study to compare methods of adjusting for stratification variables affected by misclassification in the analysis of continuous outcomes when all or only some stratification errors are discovered, and when the treatment effect or treatment-by-covariate interaction effect is of interest. The data were analysed using linear regression with no adjustment, adjustment for the strata used to perform the randomisation (randomisation strata), adjustment for the strata if all errors are corrected (true strata), and adjustment for the strata after some errors are discovered and corrected (updated strata). The unadjusted model performed poorly in all settings. Adjusting for the true strata was optimal, while the relative performance of adjusting for the randomisation strata or the updated strata varied depending on the setting. As the true strata are unlikely to be known with certainty in practice, we recommend using the updated strata for adjustment and performing subgroup analyses, provided the discovery of errors is unlikely to depend on treatment group, as expected in blinded trials. Greater transparency is needed in the reporting of stratification errors and how they were addressed in the analysis.

Evaluating whether the proportional odds models to analyse ordinal outcomes in COVID-19 clinical trials is providing clinically interpretable treatment effects: A systematic review.

BACKGROUND: After an initial recommendation from the World Health Organisation, trials of patients hospitalised with COVID-19 often include an ordinal clinical status outcome, which comprises a series of ordered categorical variables, typically ranging from 'Alive and discharged from hospital' to 'Dead'. These ordinal outcomes are often analysed using a proportional odds model, which provides a common odds ratio as an overall measure of effect, which is generally interpreted as the odds ratio for being in a higher category. The common odds ratio relies on the assumption of proportional odds, which implies an identical odds ratio across all ordinal categories; however, there is generally no statistical or biological basis for which this assumption should hold; and when violated, the common odds ratio may be a biased representation of the odds ratios for particular categories within the ordinal outcome. In this study, we aimed to evaluate to what extent the common odds ratio in published COVID-19 trials differed to simple binary odds ratios for clinically important outcomes. METHODS: We conducted a systematic review of randomised trials evaluating interventions for patients hospitalised with COVID-19, which used a proportional odds model to analyse an ordinal clinical status outcome, published between January 2020 and May 2021. We assessed agreement between the common odds ratio and the odds ratio from a standard logistic regression model for three clinically important binary outcomes: 'Alive', 'Alive without mechanical ventilation', and 'Alive and discharged from hospital'. RESULTS: Sixteen randomised clinical trials, comprising 38 individual comparisons, were included in this study; of these, only 6 trials (38%) formally assessed the proportional odds assumption. The common odds ratio differed by more than 25% compared to the binary odds ratios in 55% of comparisons for the outcome 'Alive', 37% for 'Alive without mechanical ventilation', and 24% for 'Alive and discharged from hospital'. In addition, the common odds ratio systematically underestimated the odds ratio for the outcome 'Alive' by -16.8% (95% confidence interval: -28.7% to -2.9%, p = 0.02), though differences for the other outcomes were smaller and not statistically significant (-8.4% for 'Alive without mechanical ventilation' and 3.6% for 'Alive and discharged from hospital'). The common odds ratio was statistically significant for 18% of comparisons, while the binary odds ratio was significant in 5%, 16%, and 3% of comparisons for the outcomes 'Alive', 'Alive without mechanical ventilation', and 'Alive and discharged from hospital', respectively. CONCLUSION: The common odds ratio from proportional odds models often differs substantially to odds ratios from clinically important binary outcomes, and similar to composite outcomes, a beneficial common OR from a proportional odds model does not necessarily indicate a beneficial effect on the most important categories within the ordinal outcome.

Informative cluster size in cluster-randomised trials: A case study from the TRIGGER trial.

BACKGROUND: Recent work has shown that cluster-randomised trials can estimate two distinct estimands: the participant-average and cluster-average treatment effects. These can differ when participant outcomes or the treatment effect depends on the cluster size (termed informative cluster size). In this case, estimators that target one estimand (such as the analysis of unweighted cluster-level summaries, which targets the cluster-average effect) may be biased for the other. Furthermore, commonly used estimators such as mixed-effects models or generalised estimating equations with an exchangeable correlation structure can be biased for both estimands. However, there has been little empirical research into whether informative cluster size is likely to occur in practice. METHOD: We re-analysed a cluster-randomised trial comparing two different thresholds for red blood cell transfusion in patients with acute upper gastrointestinal bleeding to explore whether estimates for the participant- and cluster-average effects differed, to provide empirical evidence for whether informative cluster size may be present. For each outcome, we first estimated a participant-average effect using independence estimating equations, which are unbiased under informative cluster size. We then compared this to two further methods: (1) a cluster-average effect estimated using either weighted independence estimating equations or unweighted cluster-level summaries, and (2) estimates from a mixed-effects model or generalised estimating equations with an exchangeable correlation structure. We then performed a small simulation study to evaluate whether observed differences between cluster- and participant-average estimates were likely to occur even if no informative cluster size was present. RESULTS: For most outcomes, treatment effect estimates from different methods were similar. However, differences of >10% occurred between participant- and cluster-average estimates for 5 of 17 outcomes (29%). We also observed several notable differences between estimates from mixed-effects models or generalised estimating equations with an exchangeable correlation structure and those based on independence estimating equations. For example, for the EQ-5D VAS score, the independence estimating equation estimate of the participant-average difference was 4.15 (95% confidence interval: -3.37 to 11.66), compared with 2.84 (95% confidence interval: -7.37 to 13.04) for the cluster-average independence estimating equation estimate, and 3.23 (95% confidence interval: -6.70 to 13.16) from a mixed-effects model. Similarly, for thromboembolic/ischaemic events, the independence estimating equation estimate for the participant-average odds ratio was 0.43 (95% confidence interval: 0.07 to 2.48), compared with 0.33 (95% confidence interval: 0.06 to 1.77) from the cluster-average estimator. CONCLUSION: In this re-analysis, we found that estimates from the various approaches could differ, which may be due to the presence of informative cluster size. Careful consideration of the estimand and the plausibility of assumptions underpinning each estimator can help ensure an appropriate analysis methods are used. Independence estimating equations and the analysis of cluster-level summaries (with appropriate weighting for each to correspond to either the participant-average or cluster-average treatment effect) are a desirable choice when informative cluster size is deemed possible, due to their unbiasedness in this setting.

Using modified intention-to-treat as a principal stratum estimator for failure to initiate treatment.

BACKGROUND: A common intercurrent event affecting many trials is when some participants do not begin their assigned treatment. For example, in a double-blind drug trial, some participants may not receive any dose of study medication. Many trials use a 'modified intention-to-treat' approach, whereby participants who do not initiate treatment are excluded from the analysis. However, it is not clear (a) the estimand being targeted by such an approach and (b) the assumptions necessary for such an approach to be unbiased. METHODS: Using potential outcome notation, we demonstrate that a modified intention-to-treat analysis which excludes participants who do not begin treatment is estimating a principal stratum estimand (i.e. the treatment effect in the subpopulation of participants who would begin treatment, regardless of which arm they were assigned to). The modified intention-to-treat estimator is unbiased for the principal stratum estimand under the assumption that the intercurrent event is not affected by the assigned treatment arm, that is, participants who initiate treatment in one arm would also do so in the other arm (i.e. if someone began the intervention, they would also have begun the control, and vice versa). RESULTS: We identify two key criteria in determining whether the modified intention-to-treat estimator is likely to be unbiased: first, we must be able to measure the participants in each treatment arm who experience the intercurrent event, and second, the assumption that treatment allocation will not affect whether the participant begins treatment must be reasonable. Most double-blind trials will satisfy these criteria, as the decision to start treatment cannot be influenced by the allocation, and we provide an example of an open-label trial where these criteria are likely to be satisfied as well, implying that a modified intention-to-treat analysis which excludes participants who do not begin treatment is an unbiased estimator for the principal stratum effect in these settings. We also give two examples where these criteria will not be satisfied (one comparing an active intervention vs usual care, where we cannot identify which usual care participants would have initiated the active intervention, and another comparing two active interventions in an unblinded manner, where knowledge of the assigned treatment arm may affect the participant's choice to begin or not), implying that a modified intention-to-treat estimator will be biased in these settings. CONCLUSION: A modified intention-to-treat analysis which excludes participants who do not begin treatment can be an unbiased estimator for the principal stratum estimand. Our framework can help identify when the assumptions for unbiasedness are likely to hold, and thus whether modified intention-to-treat is appropriate or not.

Reporting of Factorial Randomized Trials: Extension of the CONSORT 2010 Statement.

Importance: Transparent reporting of randomized trials is essential to facilitate critical appraisal and interpretation of results. Factorial trials, in which 2 or more interventions are assessed in the same set of participants, have unique methodological considerations. However, reporting of factorial trials is suboptimal. Objective: To develop a consensus-based extension to the Consolidated Standards of Reporting Trials (CONSORT) 2010 Statement for factorial trials. Design: Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the CONSORT extension for factorial trials was developed by (1) generating a list of reporting recommendations for factorial trials using a scoping review of methodological articles identified using a MEDLINE search (from inception to May 2019) and supplemented with relevant articles from the personal collections of the authors; (2) a 3-round Delphi survey between January and June 2022 to identify additional items and assess the importance of each item, completed by 104 panelists from 14 countries; and (3) a hybrid consensus meeting attended by 15 panelists to finalize the selection and wording of items for the checklist. Findings: This CONSORT extension for factorial trials modifies 16 of the 37 items in the CONSORT 2010 checklist and adds 1 new item. The rationale for the importance of each item is provided. Key recommendations are (1) the reason for using a factorial design should be reported, including whether an interaction is hypothesized, (2) the treatment groups that form the main comparisons should be clearly identified, and (3) for each main comparison, the estimated interaction effect and its precision should be reported. Conclusions and Relevance: This extension of the CONSORT 2010 Statement provides guidance on the reporting of factorial randomized trials and should facilitate greater understanding of and transparency in their reporting.

Estimands for factorial trials.

Factorial trials offer an efficient method to evaluate multiple interventions in a single trial, however the use of additional treatments can obscure research objectives, leading to inappropriate analytical methods and interpretation of results. We define a set of estimands for factorial trials, and describe a framework for applying these estimands, with the aim of clarifying trial objectives and ensuring appropriate primary and sensitivity analyses are chosen. This framework is intended for use in factorial trials where the intent is to conduct "two-trials-in-one" (ie, to separately evaluate the effects of treatments A and B), and is comprised of four steps: (i) specifying how additional treatment(s) (eg, treatment B) will be handled in the estimand, and how intercurrent events affecting the additional treatment(s) will be handled; (ii) designating the appropriate factorial estimator as the primary analysis strategy; (iii) evaluating the interaction to assess the plausibility of the assumptions underpinning the factorial estimator; and (iv) performing a sensitivity analysis using an appropriate multiarm estimator to evaluate to what extent departures from the underlying assumption of no interaction may affect results. We show that adjustment for other factors is necessary for noncollapsible effect measures (such as odds ratio), and through a trial re-analysis we find that failure to consider the estimand could lead to inappropriate interpretation of results. We conclude that careful use of the estimands framework clarifies research objectives and reduces the risk of misinterpretation of trial results, and should become a standard part of both the protocol and reporting of factorial trials.

Combining factorial and multi-arm multi-stage platform designs to evaluate multiple interventions efficiently.

BACKGROUND: Factorial designs and multi-arm multi-stage (MAMS) platform designs have many advantages, but the practical advantages and disadvantages of combining the two designs have not been explored. METHODS: We propose practical methods for a combined design within the platform trial paradigm where some interventions are not expected to interact and could be given together. RESULTS: We describe the combined design and suggest diagrams that can be used to represent it. Many properties are common both to standard factorial designs, including the need to consider interactions between interventions and the impact of intervention efficacy on power of other comparisons, and to standard multi-arm multi-stage designs, including the need to pre-specify procedures for starting and stopping intervention comparisons. We also identify some specific features of the factorial-MAMS design: timing of interim and final analyses should be determined by calendar time or total observed events; some non-factorial modifications may be useful; eligibility criteria should be broad enough to include any patient eligible for any part of the randomisation; stratified randomisation may conveniently be performed sequentially; and analysis requires special care to use only concurrent controls. CONCLUSION: A combined factorial-MAMS design can combine the efficiencies of factorial trials and multi-arm multi-stage platform trials. It allows us to address multiple research questions under one protocol and to test multiple new treatment options, which is particularly important when facing a new emergent infection such as COVID-19.

Rethinking intercurrent events in defining estimands for tuberculosis trials.

BACKGROUND/AIMS: Tuberculosis remains one of the leading causes of death from an infectious disease globally. Both choices of outcome definitions and approaches to handling events happening post-randomisation can change the treatment effect being estimated, but these are often inconsistently described, thus inhibiting clear interpretation and comparison across trials. METHODS: Starting from the ICH E9(R1) addendum's definition of an estimand, we use our experience of conducting large Phase III tuberculosis treatment trials and our understanding of the estimand framework to identify the key decisions regarding how different event types are handled in the primary outcome definition, and the important points that should be considered in making such decisions. A key issue is the handling of intercurrent (i.e. post-randomisation) events (ICEs) which affect interpretation of or preclude measurement of the intended final outcome. We consider common ICEs including treatment changes and treatment extension, poor adherence to randomised treatment, re-infection with a new strain of tuberculosis which is different from the original infection, and death. We use two completed tuberculosis trials (REMoxTB and STREAM Stage 1) as illustrative examples. These trials tested non-inferiority of new tuberculosis treatment regimens versus a control regimen. The primary outcome was a binary composite endpoint, 'favourable' or 'unfavourable', which was constructed from several components. RESULTS: We propose the following improvements in handling the above-mentioned ICEs and loss to follow-up (a post-randomisation event that is not in itself an ICE). First, changes to allocated regimens should not necessarily be viewed as an unfavourable outcome; from the patient perspective, the potential harms associated with a change in the regimen should instead be directly quantified. Second, handling poor adherence to randomised treatment using a per-protocol analysis does not necessarily target a clear estimand; instead, it would be desirable to develop ways to estimate the treatment effects more relevant to programmatic settings. Third, re-infection with a new strain of tuberculosis could be handled with different strategies, depending on whether the outcome of interest is the ability to attain culture negativity from infection with any strain of tuberculosis, or specifically the presenting strain of tuberculosis. Fourth, where possible, death could be separated into tuberculosis-related and non-tuberculosis-related and handled using appropriate strategies. Finally, although some losses to follow-up would result in early treatment discontinuation, patients lost to follow-up before the end of the trial should not always be classified as having an unfavourable outcome. Instead, loss to follow-up should be separated from not completing the treatment, which is an ICE and may be considered as an unfavourable outcome. CONCLUSION: The estimand framework clarifies many issues in tuberculosis trials but also challenges trialists to justify and improve their outcome definitions. Future trialists should consider all the above points in defining their outcomes.

Outpatient Talc Administration by Indwelling Pleural Catheter for Malignant Effusion.

BACKGROUND: Malignant pleural effusion affects more than 750,000 persons each year across Europe and the United States. Pleurodesis with the administration of talc in hospitalized patients is the most common treatment, but indwelling pleural catheters placed for drainage offer an ambulatory alternative. We examined whether talc administered through an indwelling pleural catheter was more effective at inducing pleurodesis than the use of an indwelling pleural catheter alone. METHODS: Over a period of 4 years, we recruited patients with malignant pleural effusion at 18 centers in the United Kingdom. After the insertion of an indwelling pleural catheter, patients underwent drainage regularly on an outpatient basis. If there was no evidence of substantial lung entrapment (nonexpandable lung, in which lung expansion and pleural apposition are not possible because of visceral fibrosis or bronchial obstruction) at 10 days, patients were randomly assigned to receive either 4 g of talc slurry or placebo through the indwelling pleural catheter on an outpatient basis. Talc or placebo was administered on a single-blind basis. Follow-up lasted for 70 days. The primary outcome was successful pleurodesis at day 35 after randomization. RESULTS: The target of 154 patients undergoing randomization was reached after 584 patients were approached. At day 35, a total of 30 of 69 patients (43%) in the talc group had successful pleurodesis, as compared with 16 of 70 (23%) in the placebo group (hazard ratio, 2.20; 95% confidence interval, 1.23 to 3.92; P=0.008). No significant between-group differences in effusion size and complexity, number of inpatient days, mortality, or number of adverse events were identified. No significant excess of blockages of the indwelling pleural catheter was noted in the talc group. CONCLUSIONS: Among patients without substantial lung entrapment, the outpatient administration of talc through an indwelling pleural catheter for the treatment of malignant pleural effusion resulted in a significantly higher chance of pleurodesis at 35 days than an indwelling catheter alone, with no deleterious effects. (Funded by Becton Dickinson; EudraCT number, 2012-000599-40 .).

A comparison of methods for analyzing a binary composite endpoint with partially observed components in randomized controlled trials.

Composite endpoints are commonly used to define primary outcomes in randomized controlled trials. A participant may be classified as meeting the endpoint if they experience an event in one or several components (eg, a favorable outcome based on a composite of being alive and attaining negative culture results in trials assessing tuberculosis treatments). Partially observed components that are not missing simultaneously complicate the analysis of the composite endpoint. An intuitive strategy frequently used in practice for handling missing values in the components is to derive the values of the composite endpoint from observed components when possible, and exclude from analysis participants whose composite endpoint cannot be derived. Alternatively, complete record analysis (CRA) (excluding participants with any missing components) or multiple imputation (MI) can be used. We compare a set of methods for analyzing a composite endpoint with partially observed components mathematically and by simulation, and apply these methods in a reanalysis of a published trial (TOPPS). We show that the derived composite endpoint can be missing not at random even when the components are missing completely at random. Consequently, the treatment effect estimated from the derived endpoint is biased while CRA results without the derived endpoint are valid. Missing at random mechanisms require MI of the components. We conclude that, although superficially attractive, deriving the composite endpoint from observed components should generally be avoided. Despite the potential risk of imputation model mis-specification, MI of missing components is the preferred approach in this study setting.

Independence estimators for re-randomisation trials in multi-episode settings: a simulation study.

BACKGROUND: Re-randomisation trials involve re-enrolling and re-randomising patients for each new treatment episode they experience. They are often used when interest lies in the average effect of an intervention across all the episodes for which it would be used in practice. Re-randomisation trials are often analysed using independence estimators, where a working independence correlation structure is used. However, research into independence estimators in the context of re-randomisation has been limited. METHODS: We performed a simulation study to evaluate the use of independence estimators in re-randomisation trials. We focussed on a continuous outcome, and the setting where treatment allocation does not affect occurrence of subsequent episodes. We evaluated different treatment effect mechanisms (e.g. by allowing the treatment effect to vary across episodes, or to become less effective on re-use, etc), and different non-enrolment mechanisms (e.g. where patients who experience a poor outcome are less likely to re-enrol for their second episode). We evaluated four different independence estimators, each corresponding to a different estimand (per-episode and per-patient approaches, and added-benefit and policy-benefit approaches). RESULTS: We found that independence estimators were unbiased for the per-episode added-benefit estimand in all scenarios we considered. We found independence estimators targeting other estimands (per-patient or policy-benefit) were unbiased, except when there was differential non-enrolment between treatment groups (i.e. when different types of patients from each treatment group decide to re-enrol for subsequent episodes). We found the use of robust standard errors provided close to nominal coverage in all settings where the estimator was unbiased. CONCLUSIONS: Careful choice of estimand can ensure re-randomisation trials are addressing clinically relevant questions. Independence estimators are a useful approach, and should be considered as the default estimator until the statistical properties of alternative estimators are thoroughly evaluated.

Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial.

BACKGROUND: Emergency abdominal surgery is associated with poor patient outcomes. We studied the effectiveness of a national quality improvement (QI) programme to implement a care pathway to improve survival for these patients. METHODS: We did a stepped-wedge cluster-randomised trial of patients aged 40 years or older undergoing emergency open major abdominal surgery. Eligible UK National Health Service (NHS) hospitals (those that had an emergency general surgical service, a substantial volume of emergency abdominal surgery cases, and contributed data to the National Emergency Laparotomy Audit) were organised into 15 geographical clusters and commenced the QI programme in a random order, based on a computer-generated random sequence, over an 85-week period with one geographical cluster commencing the intervention every 5 weeks from the second to the 16th time period. Patients were masked to the study group, but it was not possible to mask hospital staff or investigators. The primary outcome measure was mortality within 90 days of surgery. Analyses were done on an intention-to-treat basis. This study is registered with the ISRCTN registry, number ISRCTN80682973. FINDINGS: Treatment took place between March 3, 2014, and Oct 19, 2015. 22 754 patients were assessed for elegibility. Of 15 873 eligible patients from 93 NHS hospitals, primary outcome data were analysed for 8482 patients in the usual care group and 7374 in the QI group. Eight patients in the usual care group and nine patients in the QI group were not included in the analysis because of missing primary outcome data. The primary outcome of 90-day mortality occurred in 1210 (16%) patients in the QI group compared with 1393 (16%) patients in the usual care group (HR 1·11, 0·96-1·28). INTERPRETATION: No survival benefit was observed from this QI programme to implement a care pathway for patients undergoing emergency abdominal surgery. Future QI programmes should ensure that teams have both the time and resources needed to improve patient care. FUNDING: National Institute for Health Research Health Services and Delivery Research Programme.

How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framework.

Results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the method that provides the most favourable result (commonly referred to as 'p-hacking'). Pre-specification of the planned analysis approach is essential to help reduce such bias, as it ensures analytical methods are chosen in advance of seeing the trial data. For this reason, guidelines such as SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and ICH-E9 (International Conference for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use) require the statistical methods for a trial's primary outcome be pre-specified in the trial protocol. However, pre-specification is only effective if done in a way that does not allow p-hacking. For example, investigators may pre-specify a certain statistical method such as multiple imputation, but give little detail on how it will be implemented. Because there are many different ways to perform multiple imputation, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in different ways before deciding on a final approach. In this article, we describe a five-point framework (the Pre-SPEC framework) for designing a pre-specified analysis approach that does not allow p-hacking. This framework was designed based on the principles in the SPIRIT and ICH-E9 guidelines and is intended to be used in conjunction with these guidelines to help investigators design the statistical analysis strategy for the trial's primary outcome in the trial protocol.

Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trials.

BACKGROUND: Choosing or altering the planned statistical analysis approach after examination of trial data (often referred to as 'p-hacking') can bias the results of randomised trials. However, the extent of this issue in practice is currently unclear. We conducted a review of published randomised trials to evaluate how often a pre-specified analysis approach is publicly available, and how often the planned analysis is changed. METHODS: A review of randomised trials published between January and April 2018 in six leading general medical journals. For each trial, we established whether a pre-specified analysis approach was publicly available in a protocol or statistical analysis plan and compared this to the trial publication. RESULTS: Overall, 89 of 101 eligible trials (88%) had a publicly available pre-specified analysis approach. Only 22/89 trials (25%) had no unexplained discrepancies between the pre-specified and conducted analysis. Fifty-four trials (61%) had one or more unexplained discrepancies, and in 13 trials (15%), it was impossible to ascertain whether any unexplained discrepancies occurred due to incomplete reporting of the statistical methods. Unexplained discrepancies were most common for the analysis model (n = 31, 35%) and analysis population (n = 28, 31%), followed by the use of covariates (n = 23, 26%) and the approach for handling missing data (n = 16, 18%). Many protocols or statistical analysis plans were dated after the trial had begun, so earlier discrepancies may have been missed. CONCLUSIONS: Unexplained discrepancies in the statistical methods of randomised trials are common. Increased transparency is required for proper evaluation of results.

Treatment estimands in clinical trials of patients hospitalised for COVID-19: ensuring trials ask the right questions.

When designing a clinical trial, explicitly defining the treatment estimands of interest (that which is to be estimated) can help to clarify trial objectives and ensure the questions being addressed by the trial are clinically meaningful. There are several challenges when defining estimands. Here, we discuss a number of these in the context of trials of treatments for patients hospitalised with COVID-19 and make suggestions for how estimands should be defined for key outcomes. We suggest that treatment effects should usually be measured as differences in proportions (or risk or odds ratios) for outcomes such as death and requirement for ventilation, and differences in means for outcomes such as the number of days ventilated. We further recommend that truncation due to death should be handled differently depending on whether a patient- or resource-focused perspective is taken; for the former, a composite approach should be used, while for the latter, a while-alive approach is preferred. Finally, we suggest that discontinuation of randomised treatment should be handled from a treatment policy perspective, where non-adherence is ignored in the analysis (i.e. intention to treat).

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic.

BACKGROUND: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking. METHODS: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a 'pandemic-free world' and 'world including a pandemic' are of interest. RESULTS: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a 'pandemic-free world', participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the 'world including a pandemic', all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption - potentially incorporating a pandemic time-period indicator and participant infection status - or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses. CONCLUSIONS: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Effect of Thoracoscopic Talc Poudrage vs Talc Slurry via Chest Tube on Pleurodesis Failure Rate Among Patients With Malignant Pleural Effusions: A Randomized Clinical Trial.

Importance: Malignant pleural effusion (MPE) is challenging to manage. Talc pleurodesis is a common and effective treatment. There are no reliable data, however, regarding the optimal method for talc delivery, leading to differences in practice and recommendations. Objective: To test the hypothesis that administration of talc poudrage during thoracoscopy with local anesthesia is more effective than talc slurry delivered via chest tube in successfully inducing pleurodesis. Design, Setting, and Participants: Open-label, randomized clinical trial conducted at 17 UK hospitals. A total of 330 participants were enrolled from August 2012 to April 2018 and followed up until October 2018. Patients were eligible if they were older than 18 years, had a confirmed diagnosis of MPE, and could undergo thoracoscopy with local anesthesia. Patients were excluded if they required a thoracoscopy for diagnostic purposes or had evidence of nonexpandable lung. Interventions: Patients randomized to the talc poudrage group (n = 166) received 4 g of talc poudrage during thoracoscopy while under moderate sedation, while patients randomized to the control group (n = 164) underwent bedside chest tube insertion with local anesthesia followed by administration of 4 g of sterile talc slurry. Main Outcomes and Measures: The primary outcome was pleurodesis failure up to 90 days after randomization. Secondary outcomes included pleurodesis failure at 30 and 180 days; time to pleurodesis failure; number of nights spent in the hospital over 90 days; patient-reported thoracic pain and dyspnea at 7, 30, 90, and 180 days; health-related quality of life at 30, 90, and 180 days; all-cause mortality; and percentage of opacification on chest radiograph at drain removal and at 30, 90, and 180 days. Results: Among 330 patients who were randomized (mean age, 68 years; 181 [55%] women), 320 (97%) were included in the primary outcome analysis. At 90 days, the pleurodesis failure rate was 36 of 161 patients (22%) in the talc poudrage group and 38 of 159 (24%) in the talc slurry group (adjusted odds ratio, 0.91 [95% CI, 0.54-1.55]; P = .74; difference, -1.8% [95% CI, -10.7% to 7.2%]). No statistically significant differences were noted in any of the 24 prespecified secondary outcomes. Conclusions and Relevance: Among patients with malignant pleural effusion, thoracoscopic talc poudrage, compared with talc slurry delivered via chest tube, resulted in no significant difference in the rate of pleurodesis failure at 90 days. However, the study may have been underpowered to detect small but potentially important differences. Trial Registration: ISRCTN Identifier: ISRCTN47845793.

Development, Validation, and Comparative Assessment of an International Scoring System to Determine Risk of Upper Gastrointestinal Bleeding.

BACKGROUND & AIMS: The Glasgow-Blatchford score (GBS) and pre-endoscopy Rockall score (pRS) are used in determining prognoses of patients with acute upper gastrointestinal bleeding, but neither predicts outcomes of patients with a high level of accuracy. A scoring system is needed to identify patients at risk of adverse outcomes and patients at low risk of harm. METHODS: We pooled data from 5 data sets in Canada, the United Kingdom, and Australia on 12,711 patients with acute upper gastrointestinal bleeding. The GBS and pRS were calculated for each patient. We performed multivariable logistic regression modeling of data from 10,639 cases to develop the new scoring system Canada - United Kingdom - Adelaide (CANUKA). We performed area under the receiver operating characteristic analyses to test the ability of CANUKA to identify patients who died or had rebleeding within 30 days, surgical or radiologic intervention to control bleeding, need for therapeutic endoscopy, and transfusion-a poor outcome was defined as 1 or more of these outcomes. Patients at low risk of a poor outcome (safe for management as an outpatient) were identified based on lack of transfusion, rebleeding, therapeutic endoscopy, interventional radiology or surgery, or death. We validated in 2072 patients from a separate cohort compiled from 2 datasets. RESULTS: In the development data set there was no difference between GBS and pRS in identifying patients who died without 30 days of bleeding (area under the receiver operating characteristic curve [AUROC], 0.67; 95% CI, 0.62-0.72 for GBS; AUROC, 0.70; 95% CI, 0.66-0.74 for pRS; P = .21). The GBS was superior to the pRS in identifying patients with rebleeding, hemostatic interventions, and transfusions. In the validation data set, CANUKA had higher accuracy than the GBS in identifying patients who died within 30 days of bleeding (AUROC, 0.77 vs 0.74; P = .047), but there was no significant difference in the accuracy of these scoring systems in identifying patients who required hemostatic intervention. The GBS more accurately identified patients who required therapeutic endoscopy (AUROC, 0.78; 95% CI, 0.76-0.81 for GBS; AUROC, 0.77; 95% CI, 0.74-0.79 for CANUKA; P = .47). For patients classified as low-risk patients by CANUKA (score ≤1), 96.3% were safely discharged, whereas 16 patients with a GBS ≤1 had an adverse outcome (a 95.3% probability of safe discharge). CONCLUSIONS: In an international validation analysis of the GBS and pRS for patients with acute upper gastrointestinal bleeding, we found the GBS to more accurately identify those who later required hemostatic interventions and transfusions; the scoring systems identified 30-day mortality or rebleeding with equal levels of accuracy. We developed a scoring system (CANUKA) that had similar performance to the GBS in predicting patient outcomes and it more accurately identifies patients at low risk for adverse outcomes.