Comments on 'standard and reference-based conditional mean imputation': Regulators and trial statisticians be aware!

Accurate frequentist performance of a method is desirable in confirmatory clinical trials, but is not sufficient on its own to justify the use of a missing data method. Reference-based conditional mean imputation, with variance estimation justified solely by its frequentist performance, has the surprising and undesirable property that the estimated variance becomes smaller the greater the number of missing observations; as explained under jump-to-reference it effectively forces the true treatment effect to be exactly zero for patients with missing data.

Comment on Oberman & Vink: Should we fix or simulate the complete data in simulation studies evaluating missing data methods?

For simulation studies that evaluate methods of handling missing data, we argue that generating partially observed data by fixing the complete data and repeatedly simulating the missingness indicators is a superficially attractive idea but only rarely appropriate to use.

Eliminating Ambiguous Treatment Effects Using Estimands.

Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This is because most authors do not attempt to describe what the treatment effect represents, and instead require readers to deduce this based on the reported statistical methods. However, this approach is challenging, because many methods provide counterintuitive results. For example, some methods include data from all patients, yet the resulting treatment effect applies only to a subset of patients, whereas other methods will exclude certain patients while results will apply to everyone. Additionally, some analyses provide estimates pertaining to hypothetical settings in which patients never die or discontinue treatment. Herein we introduce estimands as a solution to the aforementioned problem. An estimand is a clear description of what the treatment effect represents, thus saving readers the necessity of trying to infer this from study methods and potentially getting it wrong. We provide examples of how estimands can remove ambiguity from reported treatment effects and describe their current use in practice. The crux of our argument is that readers should not have to infer what investigators are estimating; they should be told explicitly.

Practical approaches to Bayesian sample size determination in non-inferiority trials with binary outcomes.

Bayesian analysis of a non-inferiority trial is advantageous in allowing direct probability statements to be made about the relative treatment difference rather than relying on an arbitrary and often poorly justified non-inferiority margin. When the primary analysis will be Bayesian, a Bayesian approach to sample size determination will often be appropriate for consistency with the analysis. We demonstrate three Bayesian approaches to choosing sample size for non-inferiority trials with binary outcomes and review their advantages and disadvantages. First, we present a predictive power approach for determining sample size using the probability that the trial will produce a convincing result in the final analysis. Next, we determine sample size by considering the expected posterior probability of non-inferiority in the trial. Finally, we demonstrate a precision-based approach. We apply these methods to a non-inferiority trial in antiretroviral therapy for treatment of HIV-infected children. A predictive power approach would be most accessible in practical settings, because it is analogous to the standard frequentist approach. Sample sizes are larger than with frequentist calculations unless an informative analysis prior is specified, because appropriate allowance is made for uncertainty in the assumed design parameters, ignored in frequentist calculations. An expected posterior probability approach will lead to a smaller sample size and is appropriate when the focus is on estimating posterior probability rather than on testing. A precision-based approach would be useful when sample size is restricted by limits on recruitment or costs, but it would be difficult to decide on sample size using this approach alone.

Handling intercurrent events and missing data in non-inferiority trials using the estimand framework: A tuberculosis case study.

INTRODUCTION: The ICH E9 addendum outlining the estimand framework for clinical trials was published in 2019 but provides limited guidance around how to handle intercurrent events for non-inferiority studies. Once an estimand is defined, it is also unclear how to deal with missing values using principled analyses for non-inferiority studies. METHODS: Using a tuberculosis clinical trial as a case study, we propose a primary estimand, and an additional estimand suitable for non-inferiority studies. For estimation, multiple imputation methods that align with the estimands for both primary and sensitivity analysis are proposed. We demonstrate estimation methods using the twofold fully conditional specification multiple imputation algorithm and then extend and use reference-based multiple imputation for a binary outcome to target the relevant estimands, proposing sensitivity analyses under each. We compare the results from using these multiple imputation methods with those from the original study. RESULTS: Consistent with the ICH E9 addendum, estimands can be constructed for a non-inferiority trial which improves on the per-protocol/intention-to-treat-type analysis population previously advocated, involving respectively a hypothetical or treatment policy strategy to handle relevant intercurrent events. Results from using the 'twofold' multiple imputation approach to estimate the primary hypothetical estimand, and using reference-based methods for an additional treatment policy estimand, including sensitivity analyses to handle the missing data, were consistent with the original study's reported per-protocol and intention-to-treat analysis in failing to demonstrate non-inferiority. CONCLUSIONS: Using carefully constructed estimands and appropriate primary and sensitivity estimators, using all the information available, results in a more principled and statistically rigorous approach to analysis. Doing so provides an accurate interpretation of the estimand.

Individual participant data meta-analysis versus aggregate data meta-analysis: A case study in eczema and food allergy prevention.

INTRODUCTION: Meta-analysis traditionally uses aggregate data from published reports. Individual Participant Data (IPD) meta-analysis, which obtains and synthesizes participant-level data, is potentially more informative, but resource-intensive. The impact on the findings of meta-analyses using IPD in comparison with aggregate data has rarely been formally evaluated. METHODS: We conducted a secondary analysis of a Cochrane systematic review of skincare interventions for preventing eczema and food allergy in infants to identify the impact of the analytical choice on the review's findings. We used aggregate data meta-analysis only and contrasted the results against those of the originally published IPD meta-analysis. All meta-analysis used random effects inverse variance models. Certainty of evidence was evaluated using GRADE. RESULTS: The pooled treatment effects for the Cochrane systematic review's co-primary outcomes of eczema and food allergy were similar in IPD meta-analysis (eczema RR 1.03, 95% CI 0.81, 1.31; I2 41%, 7 studies 3075 participants), and aggregate meta-analysis (eczema RR 1.01 95% CI 0.77, 1.33; I2 53%, 7 studies, 3089 participants). In aggregate meta-analysis, the statistical heterogeneity could not be explained but using IPD it was explained by one trial which used a different, bathing intervention. For IPD meta-analysis, risk of bias was assessed as lower and more adverse event data were available compared with aggregate meta-analysis. This resulted in higher certainty of evidence, especially for adverse events. IPD meta-analysis enabled analysis of treatment interactions by age and hereditary eczema risk; and analysis of the effect of treatment adherence using pooled complier-adjusted-causal-effect analysis, none of which was possible in aggregate meta-analysis. CONCLUSIONS: For this systematic review, IPD did not significantly change primary outcome risk ratios compared with aggregate data meta-analysis. However, certainty of evidence, safety outcomes, subgroup and adherence analyses were significantly different using IPD. This demonstrates benefits of adopting an IPD approach to meta-analysis.

Skin care interventions in infants for preventing eczema and food allergy.

BACKGROUND: Eczema and food allergy are common health conditions that usually begin in early childhood and often occur in the same people. They can be associated with an impaired skin barrier in early infancy. It is unclear whether trying to prevent or reverse an impaired skin barrier soon after birth is effective for preventing eczema or food allergy. OBJECTIVES: Primary objective To assess the effects of skin care interventions such as emollients for primary prevention of eczema and food allergy in infants. Secondary objective To identify features of study populations such as age, hereditary risk, and adherence to interventions that are associated with the greatest treatment benefit or harm for both eczema and food allergy. SEARCH METHODS: We performed an updated search of the Cochrane Skin Specialised Register, CENTRAL, MEDLINE, and Embase in September 2021. We searched two trials registers in July 2021. We checked the reference lists of included studies and relevant systematic reviews, and scanned conference proceedings to identify further references to relevant randomised controlled trials (RCTs).  SELECTION CRITERIA: We included RCTs of skin care interventions that could potentially enhance skin barrier function, reduce dryness, or reduce subclinical inflammation in healthy term (> 37 weeks) infants (≤ 12 months) without pre-existing eczema, food allergy, or other skin condition. Eligible comparisons were standard care in the locality or no treatment. Types of skin care interventions could include moisturisers/emollients; bathing products; advice regarding reducing soap exposure and bathing frequency; and use of water softeners. No minimum follow-up was required. DATA COLLECTION AND ANALYSIS: This is a prospective individual participant data (IPD) meta-analysis. We used standard Cochrane methodological procedures, and primary analyses used the IPD dataset. Primary outcomes were cumulative incidence of eczema and cumulative incidence of immunoglobulin (Ig)E-mediated food allergy by one to three years, both measured at the closest available time point to two years. Secondary outcomes included adverse events during the intervention period; eczema severity (clinician-assessed); parent report of eczema severity; time to onset of eczema; parent report of immediate food allergy; and allergic sensitisation to food or inhalant allergen. MAIN RESULTS: We identified 33 RCTs comprising 25,827 participants. Of these, 17 studies randomising 5823 participants reported information on one or more outcomes specified in this review.  We included 11 studies, randomising 5217 participants, in one or more meta-analyses (range 2 to 9 studies per individual meta-analysis), with 10 of these studies providing IPD; the remaining 6 studies were included in the narrative results only.   Most studies were conducted at children's hospitals. Twenty-five studies, including all those contributing data to meta-analyses, randomised newborns up to age three weeks to receive a skin care intervention or standard infant skin care. Eight of the 11 studies contributing to meta-analyses recruited infants at high risk of developing eczema or food allergy, although the definition of high risk varied between studies. Durations of intervention and follow-up ranged from 24 hours to three years. All interventions were compared against no skin care intervention or local standard care. Of the 17 studies that reported information on our prespecified outcomes, 13 assessed emollients. We assessed most of the evidence in the review as low certainty and had some concerns about risk of bias. A rating of some concerns was most often due to lack of blinding of outcome assessors or significant missing data, which could have impacted outcome measurement but was judged unlikely to have done so. We assessed the evidence for the primary food allergy outcome as high risk of bias due to the inclusion of only one trial, where findings varied based on different assumptions about missing data. Skin care interventions during infancy probably do not change the risk of eczema by one to three years of age (risk ratio (RR) 1.03, 95% confidence interval (CI) 0.81 to 1.31; risk difference 5 more cases per 1000 infants, 95% CI 28 less to 47 more; moderate-certainty evidence; 3075 participants, 7 trials) or time to onset of eczema (hazard ratio 0.86, 95% CI 0.65 to 1.14; moderate-certainty evidence; 3349 participants, 9 trials). Skin care interventions during infancy may increase the risk of IgE-mediated food allergy by one to three years of age (RR 2.53, 95% CI 0.99 to 6.49; low-certainty evidence; 976 participants, 1 trial) but may not change risk of allergic sensitisation to a food allergen by age one to three years (RR 1.05, 95% CI 0.64 to 1.71; low-certainty evidence; 1794 participants, 3 trials). Skin care interventions during infancy may slightly increase risk of parent report of immediate reaction to a common food allergen at two years (RR 1.27, 95% CI 1.00 to 1.61; low-certainty evidence; 1171 participants, 1 trial); however, this was only seen for cow's milk, and may be unreliable due to over-reporting of milk allergy in infants. Skin care interventions during infancy probably increase risk of skin infection over the intervention period (RR 1.33, 95% CI 1.01 to 1.75; risk difference 17 more cases per 1000 infants, 95% CI one more to 38 more; moderate-certainty evidence; 2728 participants, 6 trials) and may increase the risk of infant slippage over the intervention period (RR 1.42, 95% CI 0.67 to 2.99; low-certainty evidence; 2538 participants, 4 trials) and stinging/allergic reactions to moisturisers (RR 2.24, 95% 0.67 to 7.43; low-certainty evidence; 343 participants, 4 trials), although CIs for slippages and stinging/allergic reactions were wide and include the possibility of no effect or reduced risk. Preplanned subgroup analyses showed that the effects of interventions were not influenced by age, duration of intervention, hereditary risk, filaggrin (FLG) mutation, chromosome 11 intergenic variant rs2212434, or classification of intervention type for risk of developing eczema. We could not evaluate these effects on risk of food allergy. Evidence was insufficient to show whether adherence to interventions influenced the relationship between skin care interventions and eczema or food allergy development. AUTHORS' CONCLUSIONS: Based on low- to moderate-certainty evidence, skin care interventions such as emollients during the first year of life in healthy infants are probably not effective for preventing eczema; may increase risk of food allergy; and probably increase risk of skin infection. Further study is needed to understand whether different approaches to infant skin care might prevent eczema or food allergy.

A dose-finding design for dual-agent trials with patient-specific doses for one agent with application to an opiate detoxification trial.

There is a growing interest in early phase dose-finding clinical trials studying combinations of several treatments. While the majority of dose finding designs for such setting were proposed for oncology trials, the corresponding designs are also essential in other therapeutic areas. Furthermore, there is increased recognition of recommending the patient-specific doses/combinations, rather than a single target one that would be recommended to all patients in later phases regardless of their characteristics. In this paper, we propose a dose-finding design for a dual-agent combination trial motivated by an opiate detoxification trial. The distinguishing feature of the trial is that the (continuous) dose of one compound is defined externally by the clinicians and is individual for every patient. The objective of the trial is to define the dosing function that for each patient would recommend the optimal dosage of the second compound. Via a simulation study, we have found that the proposed design results in high accuracy of individual dose recommendation and is robust to the model misspecification and assumptions on the distribution of externally defined doses.

Skincare interventions in infants for preventing eczema and food allergy: A cochrane systematic review and individual participant data meta-analysis.

OBJECTIVE: Eczema and food allergy start in infancy and have shared genetic risk factors that affect skin barrier. We aimed to evaluate whether skincare interventions can prevent eczema or food allergy. DESIGN: A prospectively planned individual participant data meta-analysis was carried out within a Cochrane systematic review to determine whether skincare interventions in term infants prevent eczema or food allergy. DATA SOURCES: Cochrane Skin Specialised Register, CENTRAL, MEDLINE, Embase and trial registries to July 2020. ELIGIBILITY CRITERIA FOR SELECTED STUDIES: Included studies were randomized controlled trials of infants <1 year with healthy skin comparing a skin intervention with a control, for prevention of eczema and food allergy outcomes between 1 and 3 years. RESULTS: Of the 33 identified trials, 17 trials (5823 participants) had relevant outcome data and 10 (5154 participants) contributed to IPD meta-analysis. Three of seven trials contributing to primary eczema analysis were at low risk of bias, and the single trial contributing to primary food allergy analysis was at high risk of bias. Interventions were mainly emollients, applied for the first 3-12 months. Skincare interventions probably do not change risk of eczema by age 1-3 years (RR 1.03, 95% CI 0.81, 1.31; I2 =41%; moderate certainty; 3075 participants, 7 trials). Sensitivity analysis found heterogeneity was explained by increased eczema in a trial of daily bathing as part of the intervention. It is unclear whether skincare interventions increase risk of food allergy by age 1-3 years (RR 2.53, 95% CI 0.99 to 6.47; very low certainty; 996 participants, 1 trial), but they probably increase risk of local skin infections (RR 1.34, 95% CI 1.02, 1.77; I2 =0%; moderate certainty; 2728 participants, 6 trials). CONCLUSION: Regular emollients during infancy probably do not prevent eczema and probably increase local skin infections.

A review of the use of controlled multiple imputation in randomised controlled trials with missing outcome data.

BACKGROUND: Missing data are common in randomised controlled trials (RCTs) and can bias results if not handled appropriately. A statistically valid analysis under the primary missing-data assumptions should be conducted, followed by sensitivity analysis under alternative justified assumptions to assess the robustness of results. Controlled Multiple Imputation (MI) procedures, including delta-based and reference-based approaches, have been developed for analysis under missing-not-at-random assumptions. However, it is unclear how often these methods are used, how they are reported, and what their impact is on trial results. This review evaluates the current use and reporting of MI and controlled MI in RCTs. METHODS: A targeted review of phase II-IV RCTs (non-cluster randomised) published in two leading general medical journals (The Lancet and New England Journal of Medicine) between January 2014 and December 2019 using MI. Data was extracted on imputation methods, analysis status, and reporting of results. Results of primary and sensitivity analyses for trials using controlled MI analyses were compared. RESULTS: A total of 118 RCTs (9% of published RCTs) used some form of MI. MI under missing-at-random was used in 110 trials; this was for primary analysis in 43/118 (36%), and in sensitivity analysis for 70/118 (59%) (3 used in both). Sixteen studies performed controlled MI (1.3% of published RCTs), either with a delta-based (n = 9) or reference-based approach (n = 7). Controlled MI was mostly used in sensitivity analysis (n = 14/16). Two trials used controlled MI for primary analysis, including one reporting no sensitivity analysis whilst the other reported similar results without imputation. Of the 14 trials using controlled MI in sensitivity analysis, 12 yielded comparable results to the primary analysis whereas 2 demonstrated contradicting results. Only 5/110 (5%) trials using missing-at-random MI and 5/16 (31%) trials using controlled MI reported complete details on MI methods. CONCLUSIONS: Controlled MI enabled the impact of accessible contextually relevant missing data assumptions to be examined on trial results. The use of controlled MI is increasing but is still infrequent and poorly reported where used. There is a need for improved reporting on the implementation of MI analyses and choice of controlled MI parameters.

Skin care interventions in infants for preventing eczema and food allergy.

BACKGROUND: Eczema and food allergy are common health conditions that usually begin in early childhood and often occur together in the same people. They can be associated with an impaired skin barrier in early infancy. It is unclear whether trying to prevent or reverse an impaired skin barrier soon after birth is effective in preventing eczema or food allergy. OBJECTIVES: Primary objective To assess effects of skin care interventions, such as emollients, for primary prevention of eczema and food allergy in infants Secondary objective To identify features of study populations such as age, hereditary risk, and adherence to interventions that are associated with the greatest treatment benefit or harm for both eczema and food allergy. SEARCH METHODS: We searched the following databases up to July 2020: Cochrane Skin Specialised Register, CENTRAL, MEDLINE, and Embase. We searched two trials registers and checked reference lists of included studies and relevant systematic reviews for further references to relevant randomised controlled trials (RCTs). We contacted field experts to identify planned trials and to seek information about unpublished or incomplete trials. SELECTION CRITERIA: RCTs of skin care interventions that could potentially enhance skin barrier function, reduce dryness, or reduce subclinical inflammation in healthy term (> 37 weeks) infants (0 to 12 months) without pre-existing diagnosis of eczema, food allergy, or other skin condition were included. Comparison was standard care in the locality or no treatment. Types of skin care interventions included moisturisers/emollients; bathing products; advice regarding reducing soap exposure and bathing frequency; and use of water softeners. No minimum follow-up was required. DATA COLLECTION AND ANALYSIS: This is a prospective individual participant data (IPD) meta-analysis. We used standard Cochrane methodological procedures, and primary analyses used the IPD dataset. Primary outcomes were cumulative incidence of eczema and cumulative incidence of immunoglobulin (Ig)E-mediated food allergy by one to three years, both measured by the closest available time point to two years. Secondary outcomes included adverse events during the intervention period; eczema severity (clinician-assessed); parent report of eczema severity; time to onset of eczema; parent report of immediate food allergy; and allergic sensitisation to food or inhalant allergen. MAIN RESULTS: This review identified 33 RCTs, comprising 25,827 participants. A total of 17 studies, randomising 5823 participants, reported information on one or more outcomes specified in this review. Eleven studies randomising 5217 participants, with 10 of these studies providing IPD, were included in one or more meta-analysis (range 2 to 9 studies per individual meta-analysis). Most studies were conducted at children's hospitals. All interventions were compared against no skin care intervention or local standard care. Of the 17 studies that reported our outcomes, 13 assessed emollients. Twenty-five studies, including all those contributing data to meta-analyses, randomised newborns up to age three weeks to receive a skin care intervention or standard infant skin care. Eight of the 11 studies contributing to meta-analyses recruited infants at high risk of developing eczema or food allergy, although definition of high risk varied between studies. Durations of intervention and follow-up ranged from 24 hours to two years. We assessed most of this review's evidence as low certainty or had some concerns of risk of bias. A rating of some concerns was most often due to lack of blinding of outcome assessors or significant missing data, which could have impacted outcome measurement but was judged unlikely to have done so. Evidence for the primary food allergy outcome was rated as high risk of bias due to inclusion of only one trial where findings varied when different assumptions were made about missing data. Skin care interventions during infancy probably do not change risk of eczema by one to two years of age (risk ratio (RR) 1.03, 95% confidence interval (CI) 0.81 to 1.31; moderate-certainty evidence; 3075 participants, 7 trials) nor time to onset of eczema (hazard ratio 0.86, 95% CI 0.65 to 1.14; moderate-certainty evidence; 3349 participants, 9 trials). It is unclear whether skin care interventions during infancy change risk of IgE-mediated food allergy by one to two years of age (RR 2.53, 95% CI 0.99 to 6.47; 996 participants, 1 trial) or allergic sensitisation to a food allergen at age one to two years (RR 0.86, 95% CI 0.28 to 2.69; 1055 participants, 2 trials) due to very low-certainty evidence for these outcomes. Skin care interventions during infancy may slightly increase risk of parent report of immediate reaction to a common food allergen at two years (RR 1.27, 95% CI 1.00 to 1.61; low-certainty evidence; 1171 participants, 1 trial). However, this was only seen for cow's milk, and may be unreliable due to significant over-reporting of cow's milk allergy in infants. Skin care interventions during infancy probably increase risk of skin infection over the intervention period (RR 1.34, 95% CI 1.02 to 1.77; moderate-certainty evidence; 2728 participants, 6 trials) and may increase risk of infant slippage over the intervention period (RR 1.42, 95% CI 0.67 to 2.99; low-certainty evidence; 2538 participants, 4 trials) or stinging/allergic reactions to moisturisers (RR 2.24, 95% 0.67 to 7.43; low-certainty evidence; 343 participants, 4 trials), although confidence intervals for slippages and stinging/allergic reactions are wide and include the possibility of no effect or reduced risk. Preplanned subgroup analyses show that effects of interventions were not influenced by age, duration of intervention, hereditary risk, FLG mutation,  or classification of intervention type for risk of developing eczema. We could not evaluate these effects on risk of food allergy. Evidence was insufficient to show whether adherence to interventions influenced the relationship between skin care interventions and risk of developing eczema or food allergy. AUTHORS' CONCLUSIONS: Skin care interventions such as emollients during the first year of life in healthy infants are probably not effective for preventing eczema, and probably increase risk of skin infection. Effects of skin care interventions on risk of food allergy are uncertain. Further work is needed to understand whether different approaches to infant skin care might promote or prevent eczema and to evaluate effects on food allergy based on robust outcome assessments.

How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framework.

Results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the method that provides the most favourable result (commonly referred to as 'p-hacking'). Pre-specification of the planned analysis approach is essential to help reduce such bias, as it ensures analytical methods are chosen in advance of seeing the trial data. For this reason, guidelines such as SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and ICH-E9 (International Conference for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use) require the statistical methods for a trial's primary outcome be pre-specified in the trial protocol. However, pre-specification is only effective if done in a way that does not allow p-hacking. For example, investigators may pre-specify a certain statistical method such as multiple imputation, but give little detail on how it will be implemented. Because there are many different ways to perform multiple imputation, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in different ways before deciding on a final approach. In this article, we describe a five-point framework (the Pre-SPEC framework) for designing a pre-specified analysis approach that does not allow p-hacking. This framework was designed based on the principles in the SPIRIT and ICH-E9 guidelines and is intended to be used in conjunction with these guidelines to help investigators design the statistical analysis strategy for the trial's primary outcome in the trial protocol.

Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trials.

BACKGROUND: Choosing or altering the planned statistical analysis approach after examination of trial data (often referred to as 'p-hacking') can bias the results of randomised trials. However, the extent of this issue in practice is currently unclear. We conducted a review of published randomised trials to evaluate how often a pre-specified analysis approach is publicly available, and how often the planned analysis is changed. METHODS: A review of randomised trials published between January and April 2018 in six leading general medical journals. For each trial, we established whether a pre-specified analysis approach was publicly available in a protocol or statistical analysis plan and compared this to the trial publication. RESULTS: Overall, 89 of 101 eligible trials (88%) had a publicly available pre-specified analysis approach. Only 22/89 trials (25%) had no unexplained discrepancies between the pre-specified and conducted analysis. Fifty-four trials (61%) had one or more unexplained discrepancies, and in 13 trials (15%), it was impossible to ascertain whether any unexplained discrepancies occurred due to incomplete reporting of the statistical methods. Unexplained discrepancies were most common for the analysis model (n = 31, 35%) and analysis population (n = 28, 31%), followed by the use of covariates (n = 23, 26%) and the approach for handling missing data (n = 16, 18%). Many protocols or statistical analysis plans were dated after the trial had begun, so earlier discrepancies may have been missed. CONCLUSIONS: Unexplained discrepancies in the statistical methods of randomised trials are common. Increased transparency is required for proper evaluation of results.

Treatment estimands in clinical trials of patients hospitalised for COVID-19: ensuring trials ask the right questions.

When designing a clinical trial, explicitly defining the treatment estimands of interest (that which is to be estimated) can help to clarify trial objectives and ensure the questions being addressed by the trial are clinically meaningful. There are several challenges when defining estimands. Here, we discuss a number of these in the context of trials of treatments for patients hospitalised with COVID-19 and make suggestions for how estimands should be defined for key outcomes. We suggest that treatment effects should usually be measured as differences in proportions (or risk or odds ratios) for outcomes such as death and requirement for ventilation, and differences in means for outcomes such as the number of days ventilated. We further recommend that truncation due to death should be handled differently depending on whether a patient- or resource-focused perspective is taken; for the former, a composite approach should be used, while for the latter, a while-alive approach is preferred. Finally, we suggest that discontinuation of randomised treatment should be handled from a treatment policy perspective, where non-adherence is ignored in the analysis (i.e. intention to treat).

Sensitivity analysis for clinical trials with missing continuous outcome data using controlled multiple imputation: A practical guide.

Missing data due to loss to follow-up or intercurrent events are unintended, but unfortunately inevitable in clinical trials. Since the true values of missing data are never known, it is necessary to assess the impact of untestable and unavoidable assumptions about any unobserved data in sensitivity analysis. This tutorial provides an overview of controlled multiple imputation (MI) techniques and a practical guide to their use for sensitivity analysis of trials with missing continuous outcome data. These include δ- and reference-based MI procedures. In δ-based imputation, an offset term, δ, is typically added to the expected value of the missing data to assess the impact of unobserved participants having a worse or better response than those observed. Reference-based imputation draws imputed values with some reference to observed data in other groups of the trial, typically in other treatment arms. We illustrate the accessibility of these methods using data from a pediatric eczema trial and a chronic headache trial and provide Stata code to facilitate adoption. We discuss issues surrounding the choice of δ in δ-based sensitivity analysis. We also review the debate on variance estimation within reference-based analysis and justify the use of Rubin's variance estimator in this setting, since as we further elaborate on within, it provides information anchored inference.

An evaluation of inverse probability weighting using the propensity score for baseline covariate adjustment in smaller population randomised controlled trials with a continuous outcome.

BACKGROUND: It is important to estimate the treatment effect of interest accurately and precisely within the analysis of randomised controlled trials. One way to increase precision in the estimate and thus improve the power for randomised trials with continuous outcomes is through adjustment for pre-specified prognostic baseline covariates. Typically covariate adjustment is conducted using regression analysis, however recently, Inverse Probability of Treatment Weighting (IPTW) using the propensity score has been proposed as an alternative method. For a continuous outcome it has been shown that the IPTW estimator has the same large sample statistical properties as that obtained via analysis of covariance. However the performance of IPTW has not been explored for smaller population trials (< 100 participants), where precise estimation of the treatment effect has potential for greater impact than in larger samples. METHODS: In this paper we explore the performance of the baseline adjusted treatment effect estimated using IPTW in smaller population trial settings. To do so we present a simulation study including a number of different trial scenarios with sample sizes ranging from 40 to 200 and adjustment for up to 6 covariates. We also re-analyse a paediatric eczema trial that includes 60 children. RESULTS: In the simulation study the performance of the IPTW variance estimator was sub-optimal with smaller sample sizes. The coverage of 95% CI's was marginally below 95% for sample sizes < 150 and ≥ 100. For sample sizes < 100 the coverage of 95% CI's was always significantly below 95% for all covariate settings. The minimum coverage obtained with IPTW was 89% with n = 40. In comparison, regression adjustment always resulted in 95% coverage. The analysis of the eczema trial confirmed discrepancies between the IPTW and regression estimators in a real life small population setting. CONCLUSIONS: The IPTW variance estimator does not perform so well with small samples. Thus we caution against the use of IPTW in small sample settings when the sample size is less than 150 and particularly when sample size < 100.

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic.

BACKGROUND: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking. METHODS: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a 'pandemic-free world' and 'world including a pandemic' are of interest. RESULTS: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a 'pandemic-free world', participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the 'world including a pandemic', all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption - potentially incorporating a pandemic time-period indicator and participant infection status - or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses. CONCLUSIONS: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Reference-based multiple imputation for missing data sensitivity analyses in trial-based cost-effectiveness analysis.

Missing data are a common issue in cost-effectiveness analysis (CEA) alongside randomised trials and are often addressed assuming the data are 'missing at random'. However, this assumption is often questionable, and sensitivity analyses are required to assess the implications of departures from missing at random. Reference-based multiple imputation provides an attractive approach for conducting such sensitivity analyses, because missing data assumptions are framed in an intuitive way by making reference to other trial arms. For example, a plausible not at random mechanism in a placebo-controlled trial would be to assume that participants in the experimental arm who dropped out stop taking their treatment and have similar outcomes to those in the placebo arm. Drawing on the increasing use of this approach in other areas, this paper aims to extend and illustrate the reference-based multiple imputation approach in CEA. It introduces the principles of reference-based imputation and proposes an extension to the CEA context. The method is illustrated in the CEA of the CoBalT trial evaluating cognitive behavioural therapy for treatment-resistant depression. Stata code is provided. We find that reference-based multiple imputation provides a relevant and accessible framework for assessing the robustness of CEA conclusions to different missing data assumptions.

May Measurement Month 2018: a pragmatic global screening campaign to raise awareness of blood pressure by the International Society of Hypertension.

AIMS: Raised blood pressure (BP) is the biggest contributor to mortality and disease burden worldwide and fewer than half of those with hypertension are aware of it. May Measurement Month (MMM) is a global campaign set up in 2017, to raise awareness of high BP and as a pragmatic solution to a lack of formal screening worldwide. The 2018 campaign was expanded, aiming to include more participants and countries. METHODS AND RESULTS: Eighty-nine countries participated in MMM 2018. Volunteers (≥18 years) were recruited through opportunistic sampling at a variety of screening sites. Each participant had three BP measurements and completed a questionnaire on demographic, lifestyle, and environmental factors. Hypertension was defined as a systolic BP ≥140 mmHg or diastolic BP ≥90 mmHg, or taking antihypertensive medication. In total, 74.9% of screenees provided three BP readings. Multiple imputation using chained equations was used to impute missing readings. 1 504 963 individuals (mean age 45.3 years; 52.4% female) were screened. After multiple imputation, 502 079 (33.4%) individuals had hypertension, of whom 59.5% were aware of their diagnosis and 55.3% were taking antihypertensive medication. Of those on medication, 60.0% were controlled and of all hypertensives, 33.2% were controlled. We detected 224 285 individuals with untreated hypertension and 111 214 individuals with inadequately treated (systolic BP ≥ 140 mmHg or diastolic BP ≥ 90 mmHg) hypertension. CONCLUSION: May Measurement Month expanded significantly compared with 2017, including more participants in more countries. The campaign identified over 335 000 adults with untreated or inadequately treated hypertension. In the absence of systematic screening programmes, MMM was effective at raising awareness at least among these individuals at risk.

Total ankle replacement : comparison of the outcomes of STAR and Mobility.

Total Ankle Replacement is a recognised treatment for end-stage ankle arthritis and an alternative to arthrodesis. This study reviews a single centre series of prospectively collected outcome measures to determine whether the Mobility performs better than the Scandinavian ankle replacement. The primary outcome measure was the survivorship. Secondary outcome measures consisted of complications and international scoring systems. 147 Scandinavian and 162 Mobility ankle replacements were reviewed at a mean follow up of 12.4 and 7.7 years respectively. The revision rate, which included liner exchange, component exchange or removal of implant was at 7 years 12.3% (18) for Scandinavian and 5.2% (8) for Mobility. The complication rate was 16.5% (22) for Scandinavian compared to 9.9 % (15) for Mobility. The results of our unit compare favourably with previous published studies. In this study the Mobility has been shown to have more favourable results at 7 years compared to the Scandinavian.