Comparison of statistical methods used to meta-analyse results from interrupted time series studies: an empirical study.

BACKGROUND: The Interrupted Time Series (ITS) is a robust design for evaluating public health and policy interventions or exposures when randomisation may be infeasible. Several statistical methods are available for the analysis and meta-analysis of ITS studies. We sought to empirically compare available methods when applied to real-world ITS data. METHODS: We sourced ITS data from published meta-analyses to create an online data repository. Each dataset was re-analysed using two ITS estimation methods. The level- and slope-change effect estimates (and standard errors) were calculated and combined using fixed-effect and four random-effects meta-analysis methods. We examined differences in meta-analytic level- and slope-change estimates, their 95% confidence intervals, p-values, and estimates of heterogeneity across the statistical methods. RESULTS: Of 40 eligible meta-analyses, data from 17 meta-analyses including 282 ITS studies were obtained (predominantly investigating the effects of public health interruptions (88%)) and analysed. We found that on average, the meta-analytic effect estimates, their standard errors and between-study variances were not sensitive to meta-analysis method choice, irrespective of the ITS analysis method. However, across ITS analysis methods, for any given meta-analysis, there could be small to moderate differences in meta-analytic effect estimates, and important differences in the meta-analytic standard errors. Furthermore, the confidence interval widths and p-values for the meta-analytic effect estimates varied depending on the choice of confidence interval method and ITS analysis method. CONCLUSIONS: Our empirical study showed that meta-analysis effect estimates, their standard errors, confidence interval widths and p-values can be affected by statistical method choice. These differences may importantly impact interpretations and conclusions of a meta-analysis and suggest that the statistical methods are not interchangeable in practice.

PRECeDe Pilot: Prevention of neonatal respiratory distress with antenatal corticosteroids before elective caesarean section in women with diabetes - a feasibility randomised trial.

OBJECTIVE: The PRECeDe Pilot Trial was designed to determine the feasibility of undertaking a multicentre, randomised controlled trial (RCT) to assess the efficacy of antenatal corticosteroids administration within 7 days before elective caesarean section (CS) in women with pre-gestational diabetes (PGDM) or gestational diabetes (GDM). DESIGN: Triple blind, parallel group, placebo-controlled, pilot RCT. SETTING: Single-centre tertiary maternity hospital in Melbourne, Australia. POPULATION: Pregnant women with PGDM (type 1 or type 2 diabetes) or GDM booked for a planned CS scheduled between 35+0 and 38+6 weeks of gestation. METHODS: Eligible participants were randomised to receive two injections of either betamethasone 11.4 mg or normal saline placebo, 24 hours apart within 7 days before CS scheduled between 35+0 and 38+6 weeks of gestation. MAIN OUTCOME MEASURE: The proportion of eligible women who consented and were randomised. TRIAL REGISTRATION: Australian and New Zealand Clinical Trials Registry ACTRN12619001475134. RESULTS: Of 537 women eligible, 182 were approached and 47 (26%) were recruited. Of these, 22 were allocated to the betamethasone group and 25 were allocated to the placebo group. There were no serious adverse events related to participation. CONCLUSION: It is feasible to undertake a triple-blind, placebo-controlled RCT investigating the efficacy of antenatal corticosteroids in preventing respiratory morbidity in infants of women with PGDM or GDM who are undergoing an elective CS between 35+0 and 38+6 weeks.

Intra-cluster correlations from the CLustered OUtcome Dataset bank to inform the design of longitudinal cluster trials.

BACKGROUND: Sample size calculations for longitudinal cluster randomised trials, such as crossover and stepped-wedge trials, require estimates of the assumed correlation structure. This includes both within-period intra-cluster correlations, which importantly differ from conventional intra-cluster correlations by their dependence on period, and also cluster autocorrelation coefficients to model correlation decay. There are limited resources to inform these estimates. In this article, we provide a repository of correlation estimates from a bank of real-world clustered datasets. These are provided under several assumed correlation structures, namely exchangeable, block-exchangeable and discrete-time decay correlation structures. METHODS: Longitudinal studies with clustered outcomes were collected to form the CLustered OUtcome Dataset bank. Forty-four available continuous outcomes from 29 datasets were obtained and analysed using each correlation structure. Patterns of within-period intra-cluster correlation coefficient and cluster autocorrelation coefficients were explored by study characteristics. RESULTS: The median within-period intra-cluster correlation coefficient for the discrete-time decay model was 0.05 (interquartile range: 0.02-0.09) with a median cluster autocorrelation of 0.73 (interquartile range: 0.19-0.91). The within-period intra-cluster correlation coefficients were similar for the exchangeable, block-exchangeable and discrete-time decay correlation structures. Within-period intra-cluster correlation coefficients and cluster autocorrelations were found to vary with the number of participants per cluster-period, the period-length, type of cluster (primary care, secondary care, community or school) and country income status (high-income country or low- and middle-income country). The within-period intra-cluster correlation coefficients tended to decrease with increasing period-length and slightly decrease with increasing cluster-period sizes, while the cluster autocorrelations tended to move closer to 1 with increasing cluster-period size. Using the CLustered OUtcome Dataset bank, an RShiny app has been developed for determining plausible values of correlation coefficients for use in sample size calculations. DISCUSSION: This study provides a repository of intra-cluster correlations and cluster autocorrelations for longitudinal cluster trials. This can help inform sample size calculations for future longitudinal cluster randomised trials.

Investigation of bias due to selective inclusion of study effect estimates in meta-analyses of nutrition research.

We aimed to explore, in a sample of systematic reviews (SRs) with meta-analyses of the association between food/diet and health-related outcomes, whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health-related outcomes published between January 2018 and June 2019. We selected the first presented meta-analysis in each review (index meta-analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta-analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta-analytic estimate to the median of a randomly constructed distribution of meta-analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty-nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42-0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta-analytic effect estimates were similar, on average, to what we would expect to see in meta-analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses, which can help readers understand the risk of selective inclusion bias in the SRs.

Variation observed in consensus judgments between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research.

OBJECTIVES: To evaluate the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research using the Risk Of Bias due to Missing Evidence (ROB-ME) tool and determine inter-rater agreement in assessments. STUDY DESIGN AND SETTING: We assembled a random sample of 42 meta-analyses of nutrition research. Eight assessors were randomly assigned to one of four pairs. Each pair assessed 21 randomly assigned meta-analyses, and each meta-analysis was assessed by two pairs. We calculated raw percentage agreement and chance corrected agreement using Gwet's Agreement Coefficient (AC) in consensus judgments between pairs. RESULTS: Across the eight signaling questions in the ROB-ME tool, raw percentage agreement ranged from 52% to 100%, and Gwet's AC ranged from 0.39 to 0.76. For the risk-of-bias judgment, the raw percentage agreement was 76% (95% confidence interval 60% to 92%) and Gwet's AC was 0.47 (95% confidence interval 0.14 to 0.80). In seven (17%) meta-analyses, either one or both pairs judged the risk of bias due to missing evidence as "low risk". CONCLUSION: Our findings indicated substantial variation in assessments in consensus judgments between pairs for the signaling questions and overall risk-of-bias judgments. More tutorials and training are needed to help researchers apply the ROB-ME tool more consistently.

Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: A simulation study.

Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

Effect estimates can be accurately calculated with data digitally extracted from interrupted time series graphs.

Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide raw data for re-analysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown. Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph was extracted by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in estimates of interruption effects (and associated statistics). Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from non-inclusion.

Methodological systematic review recommends improvements to conduct and reporting when meta-analyzing interrupted time series studies.

OBJECTIVES: Interrupted Time Series (ITS) are a type of nonrandomized design commonly used to evaluate public health policy interventions, and the impact of exposures, at the population level. Meta-analysis may be used to combine results from ITS across studies (in the context of systematic reviews) or across sites within the same study. We aimed to examine the statistical approaches, methods, and completeness of reporting in reviews that meta-analyze results from ITS. STUDY DESIGN AND SETTINGS: Eight electronic databases were searched to identify reviews (published 2000-2019) that meta-analyzed at least two ITS. Characteristics of the included reviews, the statistical methods used to analyze the ITS and meta-analyze their results, effect measures, and risk of bias assessment tools were extracted. RESULTS: Of the 4213 identified records, 54 reviews were included. Nearly all reviews (94%) used two-stage meta-analysis, most commonly fitting a random effects model (69%). Among the 41 reviews that re-analyzed the ITS, linear regression (39%) and ARIMA (20%) were most commonly used; 38% adjusted for autocorrelation. The most common effect measure meta-analyzed was an immediate level-change (46/54). Reporting of the statistical methods and ITS characteristics was often incomplete. CONCLUSION: Improvement is needed in the conduct and reporting of reviews that meta-analyze results from ITS.

Methods used to select results to include in meta-analyses of nutrition research: A meta-research study.

OBJECTIVES: To investigate how often review authors encounter multiple results from included studies that are eligible for inclusion in a particular meta-analysis, and how often methods to select results are specified. METHODS: MEDLINE and Epistemonikos were searched (January 2018-June 2019) to identify systematic reviews with meta-analysis of the association between food/diet and health-related outcomes. A random sample of these reviews was selected, and for the first presented (index) meta-analysis, rules used to select effect estimates to include in this meta-analysis were extracted from the reviews and their protocols. All effect estimates from the primary studies that were eligible for inclusion in the index meta-analyses were extracted (e.g., when a study report presented effect estimates for blood pressure at 3 weeks and 6 weeks, both unadjusted and adjusted for covariates, and all were eligible for inclusion in a meta-analysis of the effect of red meat consumption on blood pressure, we extracted all estimates, and classified the study as having "multiplicity of results"). RESULTS: Forty-two systematic reviews with 325 studies (104 randomized, 221 non-randomized) were included; 14 reviews had a protocol. In 29% of review protocols and 69% of reviews, authors specified at least one decision rule to select effect estimates when multiple were available. In 68% of studies included in the index meta-analyses, there was at least one type of multiplicity of results. CONCLUSIONS: Authors of systematic reviews of nutrition studies should anticipate encountering multiplicity of results in the included primary studies. Specification of methods to handle multiplicity when designing reviews is therefore recommended.

Creating effective interrupted time series graphs: Review and recommendations.

INTRODUCTION: Interrupted Time Series (ITS) studies may be used to assess the impact of an interruption, such as an intervention or exposure. The data from such studies are particularly amenable to visual display and, when clearly depicted, can readily show the short- and long-term impact of an interruption. Further, well-constructed graphs allow data to be extracted using digitizing software, which can facilitate their inclusion in systematic reviews and meta-analyses. AIM: We provide recommendations for graphing ITS data, examine the properties of plots presented in ITS studies, and provide examples employing our recommendations. METHODS AND RESULTS: Graphing recommendations from seminal data visualization resources were adapted for use with ITS studies. The adapted recommendations cover plotting of data points, trend lines, interruptions, additional lines and general graph components. We assessed whether 217 graphs from recently published (2013-2017) ITS studies met our recommendations and found that 130 graphs (60%) had clearly distinct data points, 100 (46%) had trend lines, and 161 (74%) had a clearly defined interruption. Accurate data extraction (requiring distinct points that align with axis tick marks and labels that allow the points to be interpreted) was possible in only 72 (33%) graphs. CONCLUSION: We found that many ITS graphs did not meet our recommendations and could be improved with simple changes. Our proposed recommendations aim to achieve greater standardization and improvement in the display of ITS data, and facilitate re-use of the data in systematic reviews and meta-analyses.

Methods used to meta-analyse results from interrupted time series studies: A methodological systematic review protocol.

Background: Systematic reviews are used to inform healthcare decision making. In reviews that aim to examine the effects of organisational, policy change or public health interventions, or exposures, evidence from interrupted time series (ITS) studies may be included. A core component of many systematic reviews is meta-analysis, which is the statistical synthesis of results across studies. There is currently a lack of guidance informing the choice of meta-analysis methods for combining results from ITS studies, and there have been no studies examining the meta-analysis methods used in practice. This study therefore aims to describe current meta-analysis methods used in a cohort of reviews of ITS studies. Methods: We will identify the 100 most recent reviews (published between 1 January 2000 and 11 October 2019) that include meta-analyses of ITS studies from a search of eight electronic databases covering several disciplines (public health, psychology, education, economics). Study selection will be undertaken independently by two authors. Data extraction will be undertaken by one author, and for a random sample of the reviews, two authors. From eligible reviews we will extract details at the review level including discipline, type of interruption and any tools used to assess the risk of bias / methodological quality of included ITS studies; at the meta-analytic level we will extract type of outcome, effect measure(s), meta-analytic methods, and any methods used to re-analyse the individual ITS studies. Descriptive statistics will be used to summarise the data. Conclusions: This review will describe the methods used to meta-analyse results from ITS studies. Results from this review will inform future methods research examining how different meta-analysis methods perform, and ultimately, the development of guidance.

Genetic Manipulation of Non-pneumophila Legionella: Protocols Developed for Legionella longbeachae.

Current biomedical research into Legionnaires' disease is dominated by studies of Legionella pneumophila, largely because this pathogen is responsible for approximately 90% of clinical disease worldwide. However, in certain geographical regions, infections with non-pneumophila species are responsible for a significant proportion of diagnosed Legionnaires' disease. Understanding the pathogenesis of these non-pneumophila species of Legionella is an important step toward clinical intervention. The capacity to genetically manipulate these pathogens is essential in order to understand the genetic factors that contribute to infection and the environmental life cycle of these bacteria. The capacity to delete, mutate, and relocate genetic regions of interest allows molecular research into gene function and importance. In this chapter, methods are outlined to introduce plasmids into Legionella by electroporation. This technique is particularly useful as it is often the essential preliminary step to experiments that observe the behavior of the bacterium under altered conditions, for example, the transformation of bacteria with reporter plasmids to monitor Dot/Icm effector translocation. Electroporation is a well-established method for transformation of competent bacteria, and here specific protocols are provided, suiting a range of materials and conditions that have been successfully applied to L. longbeachae and L. dumoffii. Additionally, a homologous recombination approach to delete genetic regions of interest in L. longbeachae is outlined. The application of these techniques allows for identification of the genetic determinants of non-pneumophila Legionella virulence and for important comparative studies with other Legionella species.

Daily Walking among Commuters: A Cross-Sectional Study of Associations with Residential, Work, and Regional Accessibility in Melbourne, Australia (2012-2014).

BACKGROUND: Most research on walking for transport has focused on the walkability of residential neighborhoods, overlooking the contribution of places of work/study and the ease with which destinations outside the immediate neighborhood can be accessed, referred to as regional accessibility. OBJECTIVES: We aimed to examine if local accessibility/walkability around place of work/study and regional accessibility are independently and interactively associated with walking. METHODS: A sample of 4,913 adult commuters was derived from a household travel survey in Melbourne, Australia (2012-2014). Local accessibility was measured as the availability of destinations within an 800-m pedestrian network from homes and places of work/education using a local living index [LLI; 0-3 (low), 4-6, 7-9, and 10-12 (high) destinations]. Regional accessibility was estimated using employment opportunity, commute travel time by mode, and public transport accessibility. Every individual's potential minutes of walking for each level of exposure (observed and counter to fact) were predicted using multivariable regression models including confounders and interaction terms. For each contrast of exposure levels of interest, the corresponding within-individual differences in predicted walking were averaged across individuals to estimate marginal effects. RESULTS: High LLI at home and work/education was associated with more minutes walking than low LLI by 3.9 [95% confidence interval (CI): 2.3, 5.5] and 8.3 (95% CI: 7.3, 9.3) min, respectively, in mutually adjusted models. Across regional accessibility measures, an independent association with walking and an interactive association with LLI at work/education was observed. To take one example, the regional accessibility measure of "Jobs within 30 min by public transport" was associated with 4.3 (95% CI: 2.9, 5.7) more mins walking for high (≥30,000 jobs) compared with low (<4,000 jobs) accessibility in adjusted models. The estimated difference for high vs. low LLI (work/education) (among those with low regional accessibility) was 3.6 min (95% CI: 2.3, 4.8), while the difference for high vs. low regional accessibility (among those with low LLI) was negligible (-0.01; 95% CI: -1.2, 1.2). However, the combined effect estimate for high LLI and high regional accessibility, compared with low on both, was 12.8 min (95% CI: 11.1, 14.5), or 9.3 (95% CI: 6.7, 11.8) min/d walking more than expected based on the separate effect estimates. CONCLUSIONS: High local living (work/education) and regional accessibility, regardless of the regional accessibility measure used, are positively associated with physical activity. High exposure to both is associated with greater benefit than exposure to one or the other alone. https://doi.org/10.1289/EHP3395.