Multistep estimators of the between-study covariance matrix under the multivariate random-effects model for meta-analysis.

A wide variety of methods are available to estimate the between-study variance under the univariate random-effects model for meta-analysis. Some, but not all, of these estimators have been extended so that they can be used in the multivariate setting. We begin by extending the univariate generalised method of moments, which immediately provides a wider class of multivariate methods than was previously available. However, our main proposal is to use this new type of estimator to derive multivariate multistep estimators of the between-study covariance matrix. We then use the connection between the univariate multistep and Paule-Mandel estimators to motivate taking the limit, where the number of steps tends toward infinity. We illustrate our methodology using two contrasting examples and investigate its properties in a simulation study. We conclude that the proposed methodology is a fully viable alternative to existing estimation methods, is well suited to sensitivity analyses that explore the use of alternative estimators, and should be used instead of the existing DerSimonian and Laird-type moments based estimator in application areas where data are expected to be heterogeneous. However, multistep estimators do not seem to outperform the existing estimators when the data are more homogeneous. Advantages of the new multivariate multistep estimator include its semi-parametric nature and that it is computationally feasible in high dimensions. Our proposed estimation methods are also applicable for multivariate random-effects meta-regression, where study-level covariates are included in the model.

Associations between the EQ-5D-5L and exacerbations of chronic obstructive pulmonary disease in the ETHOS trial.

PURPOSE: Exacerbations of chronic obstructive pulmonary disease (COPD) are associated with deteriorating health and health-related quality of life (HRQoL) among people with COPD during and after events. HRQoL data are key to evaluating treatment cost-effectiveness and informing reimbursement decisions in COPD. EuroQoL 5-dimension 5-level (EQ-5D-5L) utility scores, based on various HRQoL measures, are used in economic evaluations of pharmacotherapy. These analyses estimated associations between EQ-5D-5L utility scores and exacerbations (new and previous) in patients with moderate-to-very severe COPD. METHODS: Longitudinal mixed models for repeated measures (MMRM), adjusted for time and treatment, were conducted using data from the ETHOS study (NCT02465567); models regressed EQ-5D-5L on current and past exacerbations that occurred during the study, adjusting for other patient reported outcomes and clinical factors. RESULTS: Based on the simplest covariate adjusted model (adjusted for current exacerbations and number of previous exacerbations during the study), a current moderate exacerbation was associated with an EQ-5D-5L disutility of 0.055 (95% confidence interval: 0.048, 0.062) with an additional disutility of 0.035 (0.014, 0.055) if the exacerbation was severe. After resolving, each prior exacerbation was associated with a disutility that persisted for the remainder of the study (moderate exacerbation, 0.014 [0.011, 0.016]; further disutility for severe exacerbation, 0.011 [0.003, 0.018]). CONCLUSION: An EQ-5D-5L disutility of 0.090 was associated with a current severe exacerbation in ETHOS. Our findings suggest incorporating the effects of current, recently resolved, and cumulative exacerbations into economic models when estimating benefits and costs of COPD pharmacotherapy, as exacerbations have both acute and persistent effects.

Evaluation of walking activity and gait to identify physical and mental fatigue in neurodegenerative and immune disorders: preliminary insights from the IDEA-FAST feasibility study.

BACKGROUND: Many individuals with neurodegenerative (NDD) and immune-mediated inflammatory disorders (IMID) experience debilitating fatigue. Currently, assessments of fatigue rely on patient reported outcomes (PROs), which are subjective and prone to recall biases. Wearable devices, however, provide objective and reliable estimates of gait, an essential component of health, and may present objective evidence of fatigue. This study explored the relationships between gait characteristics derived from an inertial measurement unit (IMU) and patient-reported fatigue in the IDEA-FAST feasibility study. METHODS: Participants with IMIDs and NDDs (Parkinson's disease (PD), Huntington's disease (HD), rheumatoid arthritis (RA), systemic lupus erythematosus (SLE), primary Sjogren's syndrome (PSS), and inflammatory bowel disease (IBD)) wore a lower-back IMU continuously for up to 10 days at home. Concurrently, participants completed PROs (physical fatigue (PF) and mental fatigue (MF)) up to four times a day. Macro (volume, variability, pattern, and acceleration vector magnitude) and micro (pace, rhythm, variability, asymmetry, and postural control) gait characteristics were extracted from the accelerometer data. The associations of these measures with the PROs were evaluated using a generalised linear mixed-effects model (GLMM) and binary classification with machine learning. RESULTS: Data were recorded from 72 participants: PD = 13, HD = 9, RA = 12, SLE = 9, PSS = 14, IBD = 15. For the GLMM, the variability of the non-walking bouts length (in seconds) with PF returned the highest conditional R2, 0.165, and with MF the highest marginal R2, 0.0018. For the machine learning classifiers, the highest accuracy of the current analysis was returned by the micro gait characteristics with an intrasubject cross validation method and MF as 56.90% (precision = 43.9%, recall = 51.4%). Overall, the acceleration vector magnitude, bout length variation, postural control, and gait rhythm were the most interesting characteristics for future analysis. CONCLUSIONS: Counterintuitively, the outcomes indicate that there is a weak relationship between typical gait measures and abnormal fatigue. However, factors such as the COVID-19 pandemic may have impacted gait behaviours. Therefore, further investigations with a larger cohort are required to fully understand the relationship between gait and abnormal fatigue.

Legal, mental health and psychosocial outcomes of the RePresent Games: a quasi-experimental study.

This study evaluated a pair of video games called the RePresent games that taught users how to represent themselves in civil court. A quasi-experimental study was conducted that compared 69 RePresent game users and 78 non-game users with civil legal issues across four U.S. states on legal, mental health and psychosocial outcomes over 3 months. The results revealed that RePresent game users reported greater legal knowledge, better mental health and higher quality of life than non-game users across time, and a greater rate of improvement in legal knowledge than non-game users over time. These findings suggest that gamifying education about legal procedures for the general public holds great potential in helping individuals obtain self-help legal assistance although some formal mental health treatment may be needed for many seeking legal aid.

Comment on 'statistical consideration and challenges in bridging study of personalized medicine': a modified variance for sensitivity analysis.

A pivotal clinical trial is often necessary to assess drug efficacy in the intended to use (IU) population. Ideally, patients should be enrolled based on a positive test result from a well-characterized companion diagnostic (CDx). However, the central challenge is that patients are instead recruited on the basis of a clinical trial assay (CTA) result. This challenge arises because, CTA is available at all local labs; the time delay to enable enrollment based on CDx could result in a significant proportion of patients being unable to participate, adversely affecting precision and/or bias. The difficulty is therefore that patients are recruited on the basis that their CTA result is positive (CTA+) but the goal is to assess the drug efficacy in patients positive by the companion diagnostic (CDx+). In this commentary, we will examine an apparent weakness of a variance formula that is proposed in the context of a sensitivity analysis. We will develop an alternative formula, and argue that this should be used instead.

One-stage individual participant data meta-analysis models for continuous and binary outcomes: Comparison of treatment coding options and estimation methods.

A one-stage individual participant data (IPD) meta-analysis synthesizes IPD from multiple studies using a general or generalized linear mixed model. This produces summary results (eg, about treatment effect) in a single step, whilst accounting for clustering of participants within studies (via a stratified study intercept, or random study intercepts) and between-study heterogeneity (via random treatment effects). We use simulation to evaluate the performance of restricted maximum likelihood (REML) and maximum likelihood (ML) estimation of one-stage IPD meta-analysis models for synthesizing randomized trials with continuous or binary outcomes. Three key findings are identified. First, for ML or REML estimation of stratified intercept or random intercepts models, a t-distribution based approach generally improves coverage of confidence intervals for the summary treatment effect, compared with a z-based approach. Second, when using ML estimation of a one-stage model with a stratified intercept, the treatment variable should be coded using "study-specific centering" (ie, 1/0 minus the study-specific proportion of participants in the treatment group), as this reduces the bias in the between-study variance estimate (compared with 1/0 and other coding options). Third, REML estimation reduces downward bias in between-study variance estimates compared with ML estimation, and does not depend on the treatment variable coding; for binary outcomes, this requires REML estimation of the pseudo-likelihood, although this may not be stable in some situations (eg, when data are sparse). Two applied examples are used to illustrate the findings.

Behavioural intervention for weight loss maintenance versus standard weight advice in adults with obesity: A randomised controlled trial in the UK (NULevel Trial).

BACKGROUND: Scalable weight loss maintenance (WLM) interventions for adults with obesity are lacking but vital for the health and economic benefits of weight loss to be fully realised. We examined the effectiveness and cost-effectiveness of a low-intensity technology-mediated behavioural intervention to support WLM in adults with obesity after clinically significant weight loss (≥5%) compared to standard lifestyle advice. METHODS AND FINDINGS: The NULevel trial was an open-label randomised controlled superiority trial in 288 adults recruited April 2014 to May 2015 with weight loss of ≥5% within the previous 12 months, from a pre-weight loss BMI of ≥30 kg/m2. Participants were self-selected, and the majority self-certified previous weight loss. We used a web-based randomisation system to assign participants to either standard lifestyle advice via newsletter (control arm) or a technology-mediated low-intensity behavioural WLM programme (intervention arm). The intervention comprised a single face-to-face goal-setting meeting, self-monitoring, and remote feedback on weight, diet, and physical activity via links embedded in short message service (SMS). All participants were provided with wirelessly connected weighing scales, but only participants in the intervention arm were instructed to weigh themselves daily and told that they would receive feedback on their weight. After 12 months, we measured the primary outcome, weight (kilograms), as well as frequency of self-weighing, objective physical activity (via accelerometry), psychological variables, and cost-effectiveness. The study was powered to detect a between-group weight difference of ±2.5 kg at follow-up. Overall, 264 participants (92%) completed the trial. Mean weight gain from baseline to 12 months was 1.8 kg (95% CI 0.5-3.1) in the intervention group (n = 131) and 1.8 kg (95% CI 0.6-3.0) in the control group (n = 133). There was no evidence of an effect on weight at 12 months (difference in adjusted mean weight change from baseline: -0.07 [95% CI 1.7 to -1.9], p = 0.9). Intervention participants weighed themselves more frequently than control participants and were more physically active. Intervention participants reported greater satisfaction with weight outcomes, more planning for dietary and physical activity goals and for managing lapses, and greater confidence for healthy eating, weight loss, and WLM. Potential limitations, such as the use of connected weighing study in both trial arms, the absence of a measurement of energy intake, and the recruitment from one region of the United Kingdom, are discussed. CONCLUSIONS: There was no difference in the WLM of participants who received the NULevel intervention compared to participants who received standard lifestyle advice via newsletter. The intervention affected some, but not all, process-related secondary outcomes of the trial. TRIAL REGISTRATION: This trial is registered with the ISRCTN registry (ISRCTN 14657176; registration date 20 March 2014).

Incorporating external evidence on between-trial heterogeneity in network meta-analysis.

In a network meta-analysis, between-study heterogeneity variances are often very imprecisely estimated because data are sparse, so standard errors of treatment differences can be highly unstable. External evidence can provide informative prior distributions for heterogeneity and, hence, improve inferences. We explore approaches for specifying informative priors for multiple heterogeneity variances in a network meta-analysis. First, we assume equal heterogeneity variances across all pairwise intervention comparisons (approach 1); incorporating an informative prior for the common variance is then straightforward. Models allowing unequal heterogeneity variances are more realistic; however, care must be taken to ensure implied variance-covariance matrices remain valid. We consider three strategies for specifying informative priors for multiple unequal heterogeneity variances. Initially, we choose different informative priors according to intervention comparison type and assume heterogeneity to be proportional across comparison types and equal within comparison type (approach 2). Next, we allow all heterogeneity variances in the network to differ, while specifying a common informative prior for each. We explore two different approaches to this: placing priors on variances and correlations separately (approach 3) or using an informative inverse Wishart distribution (approach 4). Our methods are exemplified through application to two network metaanalyses. Appropriate informative priors are obtained from previously published evidence-based distributions for heterogeneity. Relevant prior information on between-study heterogeneity can be incorporated into network meta-analyses, without needing to assume equal heterogeneity across treatment comparisons. The approaches proposed will be beneficial in sparse data sets and provide more appropriate intervals for treatment differences than those based on imprecise heterogeneity estimates.

Bivariate network meta-analysis for surrogate endpoint evaluation.

Surrogate endpoints are very important in regulatory decision making in healthcare, in particular if they can be measured early compared to the long-term final clinical outcome and act as good predictors of clinical benefit. Bivariate meta-analysis methods can be used to evaluate surrogate endpoints and to predict the treatment effect on the final outcome from the treatment effect measured on a surrogate endpoint. However, candidate surrogate endpoints are often imperfect, and the level of association between the treatment effects on the surrogate and final outcomes may vary between treatments. This imposes a limitation on methods which do not differentiate between the treatments. We develop bivariate network meta-analysis (bvNMA) methods, which combine data on treatment effects on the surrogate and final outcomes, from trials investigating multiple treatment contrasts. The bvNMA methods estimate the effects on both outcomes for all treatment contrasts individually in a single analysis. At the same time, they allow us to model the trial-level surrogacy patterns within each treatment contrast and treatment-level surrogacy, thus enabling predictions of the treatment effect on the final outcome either for a new study in a new population or for a new treatment. Modelling assumptions about the between-studies heterogeneity and the network consistency, and their impact on predictions, are investigated using an illustrative example in advanced colorectal cancer and in a simulation study. When the strength of the surrogate relationships varies across treatment contrasts, bvNMA has the advantage of identifying treatment comparisons for which surrogacy holds, thus leading to better predictions.

Two new approaches for the visualisation of models for network meta-analysis.

BACKGROUND: Meta-analysis is a useful tool for combining evidence from multiple studies to estimate a pooled treatment effect. An extension of meta-analysis, network meta-analysis, is becoming more commonly used as a way to simultaneously compare multiple treatments in a single analysis. Despite the variety of approaches available for presenting fitted models, ascertaining an intuitive understanding of these models is often difficult. This is especially challenging in large networks with many different treatments. Here we propose two visualisation methods, so that network meta-analysis models can be more easily interpreted. METHODS: Our methods can be used irrespective of the statistical model or the estimation method used and are grounded in network analysis. We define three types of distance measures between the treatments that contribute to the network. These three distance measures are based on 1) the estimated treatment effects, 2) their standard errors and 3) the corresponding p-values. Then, by using a suitable threshold, we categorise some treatment pairs as being "close" (short distances). Treatments that are close are regarded as "connected" in the network analysis theory. Finally, we group the treatments into communities using standard methods for network analysis. We are then able to identify which parts of the network are estimated to have similar (or different) treatment efficacy and which parts of the network are better identified. We also propose a second method using parametric bootstrapping, where a heat map is used in the visualisation. We use the software R and provide the code used. RESULTS: We illustrate our new methods using a challenging dataset containing 22 treatments, and a previously fitted model for this data. Two communities of treatments that appear to have similar efficacy are identified. Furthermore using our methods we can identify parts of the network that are better (and less well) identified. CONCLUSIONS: Our new visualisation approaches may be used by network meta-analysts to gain an intuitive understanding of the implications of their fitted models. Our visualisation methods may be used informally, to identify the most salient features of the fitted models that can then be reported, or more formally by presenting the new visualisation devices within published reports.

Identification and characterization of two novel alternatively spliced E2F1 transcripts in the rat CNS.

E2F1 is a transcription factor classically known to regulate G0/G1 to S phase progression in the cell cycle. In addition, E2F1 also regulates a wide range of apoptotic genes and thus has been well studied in the context of neuronal death and neurodegenerative diseases. However, its function and regulation in the mature central nervous system are not well understood. Alternative splicing is a well-conserved post-transcriptional mechanism common in cells of the CNS and is necessary to generate diverse functional modifications to RNA or protein products from genes. Heretofore, physiologically significant alternatively spliced E2F1 transcripts have not been reported. In the present study, we report the identification of two novel alternatively spliced E2F1 transcripts: E2F1b, an E2F1 transcript retaining intron 5, and E2F1c, an E2F1 transcript excluding exon 6. These alternatively spliced transcripts are observed in the brain and neural cell types including neurons, astrocytes, and undifferentiated oligodendrocytes. The expression of these E2F1 transcripts is distinct during maturation of primary hippocampal neuroglial cells. Pharmacologically-induced global translation inhibition with cycloheximide, anisomycin or thapsigargin lead to significantly reduced expression of E2F1a, E2F1b and E2F1c. Conversely, increasing neuronal activity by elevating the concentration of potassium chloride selectively increased the expression of E2F1b. Furthermore, experiments expressing these variants in vitro show the transcripts can be translated to generate a protein product. Taken together, our data suggest that the alternatively spliced E2F1 transcript behave differently than the E2F1a transcript, and our results provide a foundation for future investigation of the function of E2F1 splice variants in the CNS.

Feasibility of parent communication training with remote coaching using smartphone apps.

BACKGROUND: Communication training for parents of young children with neurodisability is often delivered in groups and includes video coaching. Group teaching is problematic when there is wide variation in the characteristics and needs amongst participants. AIMS: To assess the potential feasibility and acceptability of delivering one-to-one parent training supported by remote coaching using smartphone apps and of conducting further trials of the intervention. METHODS & PROCEDURES: We aimed to recruit eight children aged 12-48 months with motor disorders and communication difficulties and to provide families with individual parent training in six weekly home visits supplemented by remote coaching via smartphone apps. For outcome measurement, parents recorded their interaction with their child thrice weekly during baseline (3 weeks), intervention, post-intervention (3 weeks) and follow-up (1 week). Measures comprised parent responsiveness and counts of children's communication and vocalization. Research design feasibility was measured through rates of recruitment, attrition, outcome measure completion and agreement between raters on outcome measurement. Intervention feasibility was assessed through the proportion of therapy sessions received, the number of videos and text messages shared using the apps in remote coaching, and message content. Parents were interviewed about the acceptability of the intervention and trial design. Interviews were transcribed and analyzed using inductive thematic analysis. OUTCOMES & RESULTS: Nine children were recruited over 16 weeks. All fitted the inclusion criteria. Four families withdrew from the study. Five families completed the intervention. No family submitted the target number of video recordings for outcome measurement. Interrater agreement was moderate for child communication (K = 0.46) and vocalization (K = 0.60) and high for The Responsive Augmentative and Alternative Communication Style scale (RAACS) (rs = 0.96). Parents who completed the intervention reported positive experiences of the programme and remote coaching via the apps. Therapist messages via the app contained comments on parent and child behaviour and requests for parental reflection/action; parental messages contained reflections on children's communication. CONCLUSIONS & IMPLICATIONS: The intervention and study design demanded high levels of parental involvement and was not suitable for all families. Recording shorter periods of interaction via mobile phones or using alternative methods of data collection may increase feasibility of outcome measurement.

Discussion on Quantifying publication bias in meta-analysis.

In this discussion, I will describe some issues that are related to the article presented by Lin and Chu. In particular, I discuss three concerns that should be addressed before their methodology may be accepted for general use.

A matrix-based method of moments for fitting multivariate network meta-analysis models with multiple outcomes and random inconsistency effects.

Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here, we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent between-study heterogeneity and inconsistency. Inconsistency arises when different forms of direct and indirect evidence are not in agreement, even having taken between-study heterogeneity into account. However, the consistency assumption is often assumed in practice and so we also explain how to fit a reduced model which makes this assumption. Our estimation method extends several other commonly used methods for meta-analysis, including the method proposed by DerSimonian and Laird (). We investigate the use of our proposed methods in the context of both a simulation study and a real example.

Multistep estimators of the between-study variance: The relationship with the Paule-Mandel estimator.

A wide variety of estimators of the between-study variance are available in random-effects meta-analysis. Many, but not all, of these estimators are based on the method of moments. The DerSimonian-Laird estimator is widely used in applications, but the Paule-Mandel estimator is an alternative that is now recommended. Recently, DerSimonian and Kacker have developed two-step moment-based estimators of the between-study variance. We extend these two-step estimators so that multiple (more than two) steps are used. We establish the surprising result that the multistep estimator tends towards the Paule-Mandel estimator as the number of steps becomes large. Hence, the iterative scheme underlying our new multistep estimator provides a hitherto unknown relationship between two-step estimators and Paule-Mandel estimator. Our analysis suggests that two-step estimators are not necessarily distinct estimators in their own right; instead, they are quantities that are closely related to the usual iterative scheme that is used to calculate the Paule-Mandel estimate. The relationship that we establish between the multistep and Paule-Mandel estimator is another justification for the use of the latter estimator. Two-step and multistep estimators are perhaps best conceptualized as approximate Paule-Mandel estimators.

A comparison of seven random-effects models for meta-analyses that estimate the summary odds ratio.

Comparative trials that report binary outcome data are commonly pooled in systematic reviews and meta-analyses. This type of data can be presented as a series of 2-by-2 tables. The pooled odds ratio is often presented as the outcome of primary interest in the resulting meta-analysis. We examine the use of 7 models for random-effects meta-analyses that have been proposed for this purpose. The first of these models is the conventional one that uses normal within-study approximations and a 2-stage approach. The other models are generalised linear mixed models that perform the analysis in 1 stage and have the potential to provide more accurate inference. We explore the implications of using these 7 models in the context of a Cochrane Review, and we also perform a simulation study. We conclude that generalised linear mixed models can result in better statistical inference than the conventional 2-stage approach but also that this type of model presents issues and difficulties. These challenges include more demanding numerical methods and determining the best way to model study specific baseline risks. One possible approach for analysts is to specify a primary model prior to performing the systematic review but also to present the results using other models in a sensitivity analysis. Only one of the models that we investigate is found to perform poorly so that any of the other models could be considered for either the primary or the sensitivity analysis.

Editor's Choice - The Implications of Non-compliance to Endovascular Aneurysm Repair Surveillance: A Systematic Review and Meta-analysis.

OBJECTIVE/BACKGROUND: Increasingly, reports show that compliance rates with endovascular aneurysm repair (EVAR) surveillance are often suboptimal. The aim of this study was to determine the safety implications of non-compliance with surveillance. METHODS: The study was carried out according to the Preferred Items for Reporting of Systematic Reviews and Meta-Analyses (PRISMA) guidelines. An electronic search was undertaken by two independent authors using Embase, MEDLINE, Cochrane, and Web of Science databases from 1990 to July 2017. Only studies that analysed infrarenal EVAR and had a definition of non-compliance described as weeks or months without imaging surveillance were analysed. Meta-analysis was carried out using the random-effects model and restricted maximum likelihood estimation. RESULTS: Thirteen articles (40,730 patients) were eligible for systematic review; of these, seven studies (14,311 patients) were appropriate for comparative meta-analyses of mortality rates. Three studies (8316 patients) were eligible for the comparative meta-analyses of re-intervention rates after EVAR and four studies (12,995 patients) eligible for meta-analysis for abdominal aortic aneurysm related mortality (ARM). The estimated average non-compliance rate was 42.0% (95% confidence interval [CI] 28-56%). Although there is some evidence that non-compliant patients have better survival rates, there was no statistically significant difference in all cause mortality rates (year 1: odds ratio [OR] 5.77, 95% CI 0.74-45.14; year 3: OR 2.28, 95% CI 0.92-5.66; year 5: OR 1.81, 95% CI 0.88-3.74) and ARM (OR 1.47, 95% CI 0.99-2.19) between compliant and non-compliant patients in the first 5 years after EVAR. The re-intervention rate was statistically significantly higher in compliant patients from 3 to 5 years after EVAR (year 1: OR 6.36, 95% CI 0.23-172.73; year 3: OR 3.94, 85% CI 1.46-10.69; year 5: OR 5.34, 95% CI 1.87-15.29). CONCLUSION: This systematic review and meta-analysis suggests that patients compliant with EVAR surveillance programmes may have an increased re-intervention rate but do not appear to have better survival rates than non-compliant patients.

When should meta-analysis avoid making hidden normality assumptions?

Meta-analysis is a widely used statistical technique. The simplicity of the calculations required when performing conventional meta-analyses belies the parametric nature of the assumptions that justify them. In particular, the normal distribution is extensively, and often implicitly, assumed. Here, we review how the normal distribution is used in meta-analysis. We discuss when the normal distribution is likely to be adequate and also when it should be avoided. We discuss alternative and more advanced methods that make less use of the normal distribution. We conclude that statistical methods that make fewer normality assumptions should be considered more often in practice. In general, statisticians and applied analysts should understand the assumptions made by their statistical analyses. They should also be able to defend these assumptions. Our hope is that this article will foster a greater appreciation of the extent to which assumptions involving the normal distribution are made in statistical methods for meta-analysis. We also hope that this article will stimulate further discussion and methodological work.

The Hartung-Knapp modification for random-effects meta-analysis: A useful refinement but are there any residual concerns?

The modified method for random-effects meta-analysis, usually attributed to Hartung and Knapp and also proposed by Sidik and Jonkman, is easy to implement and is becoming advocated for general use. Here, we examine a range of potential concerns about the widespread adoption of this method. Motivated by these issues, a variety of different conventions can be adopted when using the modified method in practice. We describe and investigate the use of a variety of these conventions using a new taxonomy of meta-analysis datasets. We conclude that the Hartung and Knapp modification may be a suitable replacement for the standard method. Despite this, analysts who advocate the modified method should be ready to defend its use against the possible objections to it that we present. We further recommend that the results from more conventional approaches should be used as sensitivity analyses when using the modified method. It has previously been suggested that a common-effect analysis should be used for this purpose but we suggest amending this recommendation and argue that a standard random-effects analysis should be used instead.

Inference for correlated effect sizes using multiple univariate meta-analyses.

Multivariate meta-analysis, which involves jointly analyzing multiple and correlated outcomes from separate studies, has received a great deal of attention. One reason to prefer the multivariate approach is its ability to account for the dependence between multiple estimates from the same study. However, nearly all the existing methods for analyzing multivariate meta-analytic data require the knowledge of the within-study correlations, which are usually unavailable in practice. We propose a simple non-iterative method that can be used for the analysis of multivariate meta-analysis datasets, that has no convergence problems, and does not require the use of within-study correlations. Our approach uses standard univariate methods for the marginal effects but also provides valid joint inference for multiple parameters. The proposed method can directly handle missing outcomes under missing completely at random assumption. Simulation studies show that the proposed method provides unbiased estimates, well-estimated standard errors, and confidence intervals with good coverage probability. Furthermore, the proposed method is found to maintain high relative efficiency compared with conventional multivariate meta-analyses where the within-study correlations are known. We illustrate the proposed method through two real meta-analyses where functions of the estimated effects are of interest.