Multilevel mediation analysis in R: A comparison of bootstrap and Bayesian approaches.

Mediation analysis in repeated measures studies can shed light on the mechanisms through which experimental manipulations change the outcome variable. However, the literature on interval estimation for the indirect effect in the 1-1-1 single mediator model is sparse. Most simulation studies to date evaluating mediation analysis in multilevel data considered scenarios that do not match the expected numbers of level 1 and level 2 units typically encountered in experimental studies, and no study to date has compared resampling and Bayesian methods for constructing intervals for the indirect effect in this context. We conducted a simulation study to compare statistical properties of interval estimates of the indirect effect obtained using four bootstrap and two Bayesian methods in the 1-1-1 mediation model with and without random effects. Bayesian credibility intervals had coverage closest to the nominal value and no instances of excessive Type I error rates, but lower power than resampling methods. Findings indicated that the pattern of performance for resampling methods often depended on the presence of random effects. We provide suggestions for selecting an interval estimator for the indirect effect depending on the most important statistical property for a given study, as well as code in R for implementing all methods evaluated in the simulation study. Findings and code from this project will hopefully support the use of mediation analysis in experimental research with repeated measures.

Sequential Bayesian Data Synthesis for Mediation and Regression Analysis.

Science is an inherently cumulative process, and knowledge on a specific topic is organized through synthesis of findings from related studies. Meta-analysis has been the most common statistical method for synthesizing findings from multiple studies in prevention science and other fields. In recent years, Bayesian statistics have been put forth as another way to synthesize findings and have been praised for providing a natural framework for update existing knowledge with new data. This article presents a Bayesian method for cumulative science and describes a SAS macro %SBDS for synthesizing findings from multiple studies or multiple data sets from a single study using three different methods: meta-analysis using raw data, sequential Bayesian data synthesis, and a single-level analysis on pooled data. Sequential Bayesian data synthesis and Bayesian statistics in general are discussed in an accessible manner, and guidelines are provided on how researchers can use the accompanying SAS macro for synthesizing data from their own studies. Four alcohol use studies were used to demonstrate how to apply the three data synthesis methods using the SAS macro.

Bayesian Mediation Analysis with Power Prior Distributions.

Bayesian methods are often suggested as a solution for issues encountered in small sample research, however, Bayesian methods often require informative priors to outperform classical methods in these settings. Specifying accurate priors with respect to the true value of the parameter of interest is challenging and inaccurate informative priors can have detrimental effects on conclusions from the statistical analysis. This paper proposes an objective procedure for creating informative priors for mediation analysis based on a historical data set; the only requirements for implementing the procedure are that the data from the current study constitute a representative sample from the population of interest, and that the historical and current data sets contain measures of the same covariates and independent variable, mediator, and outcome. The simulation study findings show that the proposed method leads to appropriate amount of borrowing from the historical data set, which leads to increases in precision and power when the historical data and current data are exchangeable, and does not induce bias when the historical and current studies are not exchangeable. The proposed method is illustrated using data from the project PROsetta Stone, and we provide rstan code for implementing the proposed method.

Different Roles of Prior Distributions in the Single Mediator Model with Latent Variables.

In manifest variable models, Bayesian methods for mediation analysis can have better statistical properties than commonly used frequentist methods. However, with latent variables, Bayesian mediation analysis with diffuse priors can yield worse statistical properties than frequentist methods, and no study to date has evaluated the impact of informative priors on statistical properties of point and interval summaries of the mediated effect. This article describes the first examination of using fully conjugate and informative (accurate and inaccurate) priors in Bayesian mediation analysis with latent variables. Results suggest that fully conjugate priors and informative priors with the same relative prior sample sizes have notably different effects at N = 200 and 400, than at N = 50 and 100. Consequences of a small amount of inaccuracy in priors for loadings can be alleviated by making the prior less informative, whereas the same is not always true of inaccuracy in priors for structural paths. Finally, the consequences of using informative priors depend on the inferential goals of the analysis: inaccurate priors are more detrimental for accurately estimating the mediated effect than for evaluating whether the mediated effect is nonzero. Recommendations are provided about when to gainfully employ Bayesian mediation analysis with latent variables.

Testing Mediators of Youth Intervention Outcomes Using Single-Case Experimental Designs.

The major aim of this manuscript is to bring together two important topics that have recently received much attention in child and adolescent research, albeit separately from each other: single-case experimental designs and statistical mediation analysis. Single-case experimental designs (SCEDs) are increasingly recognized as a valuable alternative for Randomized Controlled Trials (RCTs) to test intervention effects in youth populations. Statistical mediation analysis helps provide understanding about the most potent mechanisms of change underlying youth intervention outcomes. In this manuscript we: (i) describe the conceptual framework and outline desiderata for methods for mediation analysis in SCEDs; (ii) describe the main aspects of several data-analytic techniques potentially useful to test mediation in SCEDs; (iii) apply these methods to a single-case treatment data set from one clinically anxious client; and (iv) discuss pros and cons of these methods for testing mediation in SCEDs, and provide future directions.

Adjustment for unmeasured confounding through informative priors for the confounder-outcome relation.

BACKGROUND: Observational studies of medical interventions or risk factors are potentially biased by unmeasured confounding. In this paper we propose a Bayesian approach by defining an informative prior for the confounder-outcome relation, to reduce bias due to unmeasured confounding. This approach was motivated by the phenomenon that the presence of unmeasured confounding may be reflected in observed confounder-outcome relations being unexpected in terms of direction or magnitude. METHODS: The approach was tested using simulation studies and was illustrated in an empirical example of the relation between LDL cholesterol levels and systolic blood pressure. In simulated data, a comparison of the estimated exposure-outcome relation was made between two frequentist multivariable linear regression models and three Bayesian multivariable linear regression models, which varied in the precision of the prior distributions. Simulated data contained information on a continuous exposure, a continuous outcome, and two continuous confounders (one considered measured one unmeasured), under various scenarios. RESULTS: In various scenarios the proposed Bayesian analysis with an correctly specified informative prior for the confounder-outcome relation substantially reduced bias due to unmeasured confounding and was less biased than the frequentist model with covariate adjustment for one of the two confounding variables. Also, in general the MSE was smaller for the Bayesian model with informative prior, compared to the other models. CONCLUSIONS: As incorporating (informative) prior information for the confounder-outcome relation may reduce the bias due to unmeasured confounding, we consider this approach one of many possible sensitivity analyses of unmeasured confounding.

Statistical properties of four effect-size measures for mediation models.

This project examined the performance of classical and Bayesian estimators of four effect size measures for the indirect effect in a single-mediator model and a two-mediator model. Compared to the proportion and ratio mediation effect sizes, standardized mediation effect-size measures were relatively unbiased and efficient in the single-mediator model and the two-mediator model. Percentile and bias-corrected bootstrap interval estimates of ab/s Y , and ab(s X )/s Y in the single-mediator model outperformed interval estimates of the proportion and ratio effect sizes in terms of power, Type I error rate, coverage, imbalance, and interval width. For the two-mediator model, standardized effect-size measures were superior to the proportion and ratio effect-size measures. Furthermore, it was found that Bayesian point and interval summaries of posterior distributions of standardized effect-size measures reduced excessive relative bias for certain parameter combinations. The standardized effect-size measures are the best effect-size measures for quantifying mediated effects.

Attention Deficit Hyperactivity Disorder, Aggression, and Illicit Stimulant Use: Is This Self-Medication?

This study compares adults with and without attention deficit hyperactivity disorder (ADHD) on measures of direct and displaced aggression and illicit drug use. Three hundred ninety-six adults were administered the Wender Utah Rating Scale, the Risk Behavior Assessment, the Aggression Questionnaire (AQ), and the Displaced Aggression Questionnaire (DAQ). Those with ADHD were higher on all scales of the AQ and DAQ, were younger at first use of amphetamines, and were more likely to have ever used crack and amphetamines. A Structural Equation Model found a significant interaction in that for those with medium and high levels of verbal aggression, ADHD predicts crack and amphetamine. Follow-up logistic regression models suggest that blacks self-medicate with crack and whites and Hispanics self-medicate with amphetamine when they have ADHD and verbal aggression.

A Note on Testing Mediated Effects in Structural Equation Models: Reconciling Past and Current Research on the Performance of the Test of Joint Significance.

Methods to assess the significance of mediated effects in education and the social sciences are well studied and fall into two categories: single sample methods and computer-intensive methods. A popular single sample method to detect the significance of the mediated effect is the test of joint significance, and a popular computer-intensive method to detect the significance of the mediated effect is the bias-corrected bootstrap method. Both these methods are used for testing the significance of mediated effects in structural equation models (SEMs). A recent study by Leth-Steensen and Gallitto 2015 provided evidence that the test of joint significance was more powerful than the bias-corrected bootstrap method for detecting mediated effects in SEMs, which is inconsistent with previous research on the topic. The goal of this article was to investigate this surprising result and describe two issues related to testing the significance of mediated effects in SEMs which explain the inconsistent results regarding the power of the test of joint significance and the bias-corrected bootstrap found by Leth-Steensen and Gallitto 2015. The first issue was that the bias-corrected bootstrap method was conducted incorrectly. The bias-corrected bootstrap was used to estimate the standard error of the mediated effect as opposed to creating confidence intervals. The second issue was that the correlation between the path coefficients of the mediated effect was ignored as an important aspect of testing the significance of the mediated effect in SEMs. The results of the replication study confirmed prior research on testing the significance of mediated effects. That is, the bias-corrected bootstrap method was more powerful than the test of joint significance, and the bias-corrected bootstrap method had elevated Type 1 error rates in some cases. Additional methods for testing the significance of mediated effects in SEMs were considered and limitations and future directions were discussed.

The Distribution of the Product Explains Normal Theory Mediation Confidence Interval Estimation.

The distribution of the product has several useful applications. One of these applications is its use to form confidence intervals for the indirect effect as the product of 2 regression coefficients. The purpose of this article is to investigate how the moments of the distribution of the product explain normal theory mediation confidence interval coverage and imbalance. Values of the critical ratio for each random variable are used to demonstrate how the moments of the distribution of the product change across values of the critical ratio observed in research studies. Results of the simulation study showed that as skewness in absolute value increases, coverage decreases. And as skewness in absolute value and kurtosis increases, imbalance increases. The difference between testing the significance of the indirect effect using the normal theory versus the asymmetric distribution of the product is further illustrated with a real data example. This article is the first study to show the direct link between the distribution of the product and indirect effect confidence intervals and clarifies the results of previous simulation studies by showing why normal theory confidence intervals for indirect effects are often less accurate than those obtained from the asymmetric distribution of the product or from resampling methods.

Bayesian mediation analysis in trauma research.

OBJECTIVE: Bayesian methods are growing in popularity among social scientists, due to the significant advantages offered to researchers: namely, intuitive probabilistic interpretations of results. Here, we highlight the benefits of using the Bayesian framework in research where collecting large samples is challenging, specifically: the absence of a requirement of large samples for convergence, and the possibility of building on prior research by including informative priors. METHOD: We demonstrate how to fit a single mediator model and impute missing data in the Bayesian framework using the software JAGS via the R package rjags. To this end, we use open-access data to fit a mediation model and calculate the posterior probability that the mediated effect is above a specified criterion. RESULTS: We replicate the results of the original paper in the Bayesian framework and provide annotated code for mediation analysis in rjags, as well as two additional R packages for Bayesian analysis (brms and rstan) and two additional software packages (SAS and Mplus). CONCLUSION: We provide guidelines for reporting and interpreting results obtained in the Bayesian framework, and two extensions to the mediation model are discussed: adding covariates to the model and selecting informative priors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Causal mediation effects in single case experimental designs.

Single case experimental designs (SCEDs) are used to test treatment effects in a wide range of fields and consist of repeated measurements for a single case throughout one or more baseline phases and throughout one or more treatment phases. Recently, mediation analysis has been applied to SCEDs. Mediation analysis decomposes the total treatment-outcome effect into a direct and indirect effect, and therefore aims to unravel the causal processes underlying treatment-outcome effects. The most recent methodological advancement for mediation analysis is the development of causal mediation analysis methodology which clarifies the necessary causal assumptions for mediation analysis. The goal of this article is to derive the causal mediation effects and corresponding standard errors based on piecewise linear regression models for the mediator and outcome and to evaluate the performance of these regression estimators and standard errors. Whereas previous studies estimated the direct and indirect effects as either the change in level or change in trend, we showed that the causal direct and indirect effects incorporate both the change in level and change in trend. Based on our simulation study we showed that for the causal indirect effects, Monte Carlo confidence intervals provided accurate (i.e., p = .05) Type I error rates and higher statistical power than normal theory confidence intervals. For the causal direct effects and total effect, normal theory confidence intervals provided accurate Type I error rates and higher statistical power than the Monte Carlo confidence intervals. Limitations and future directions are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Multilevel analysis of matching behavior: A comparison of maximum likelihood and Bayesian estimation.

While trying to infer laws of behavior, accounting for both within-subjects and between-subjects variance is often overlooked. It has been advocated recently to use multilevel modeling to analyze matching behavior. Using multilevel modeling within behavior analysis has its own challenges though. Adequate sample sizes are required (at both levels) for unbiased parameter estimates. The purpose of the current study is to compare parameter recovery and hypothesis rejection rates of maximum likelihood (ML) estimation and Bayesian estimation (BE) of multilevel models for matching behavior studies. Four factors were investigated through simulations: number of subjects, number of measurements by subject, sensitivity (slope), and variance of the random effect. Results showed that both ML estimation and BE with flat priors yielded acceptable statistical properties for intercept and slope fixed effects. The ML estimation procedure generally had less bias, lower RMSE, more power, and false-positive rates closer to the nominal rate. Thus, we recommend ML estimation over BE with uninformative priors, considering our results. The BE procedure requires more informative priors to be used in multilevel modeling of matching behavior, which will require further studies.

Do Childcare Teachers Evaluate Children's Weight Status More Accurately Than Parents? A Brief Report.

PURPOSE: Parents' underestimation of young children's weight can reduce their engagement and readiness to implement changes in children's diet and physical activity. Childcare teachers can support parents' identification of children at risk for being overweight only if they can accurately do this themselves. DESIGN: Quantitative, cross-sectional study. SETTING: Fifteen kindergartens near Lisbon, Portugal. SUBJECTS: 319 parents, 32 teachers (47.5% and 100% response rate, respectively), and 319 children. MEASURES: Caregivers classified the children's weight, considering their height and age as underweight, healthy weight, or overweight; children's body mass index (BMI) status for age and sex was assessed. ANALYSIS: Differences in caregivers' accuracy of children's weight perception were assessed. Multilevel multivariate logistic regression models were used to analyze the predictors of the accuracy of teachers' and parents' weight perception as a binary outcome. RESULTS: The proportion of children with overweight correctly assessed differed significantly (P = 0.004) between teachers (31.1%) and parents (17.5%). The child's BMI percentile was the only significant positive predictor for both caregivers' weight perception accuracy (P < 0.001 and P = 0.004, for parents and teachers, respectively), holding the child's age and sex constant. CONCLUSION: Although childcare teachers were better raters than parents when evaluating children's weight status, the percentage of children with overweight that childcare teachers misclassified was still relatively high.

Life course of retrospective harmonization initiatives: key elements to consider.

Optimizing research on the developmental origins of health and disease (DOHaD) involves implementing initiatives maximizing the use of the available cohort study data; achieving sufficient statistical power to support subgroup analysis; and using participant data presenting adequate follow-up and exposure heterogeneity. It also involves being able to undertake comparison, cross-validation, or replication across data sets. To answer these requirements, cohort study data need to be findable, accessible, interoperable, and reusable (FAIR), and more particularly, it often needs to be harmonized. Harmonization is required to achieve or improve comparability of the putatively equivalent measures collected by different studies on different individuals. Although the characteristics of the research initiatives generating and using harmonized data vary extensively, all are confronted by similar issues. Having to collate, understand, process, host, and co-analyze data from individual cohort studies is particularly challenging. The scientific success and timely management of projects can be facilitated by an ensemble of factors. The current document provides an overview of the 'life course' of research projects requiring harmonization of existing data and highlights key elements to be considered from the inception to the end of the project.

Methods for Modeling Autocorrelation and Handling Missing Data in Mediation Analysis in Single Case Experimental Designs (SCEDs).

Single-Case Experimental Designs (SCEDs) are increasingly recognized as a valuable alternative to group designs. Mediation analysis is useful in SCEDs contexts because it informs researchers about the underlying mechanism through which an intervention influences the outcome. However, methods for conducting mediation analysis in SCEDs have only recently been proposed. Furthermore, repeated measures of a target behavior present the challenges of autocorrelation and missing data. This paper aims to extend methods for estimating indirect effects in piecewise regression analysis in SCEDs by (1) evaluating three methods for modeling autocorrelation, namely, Newey-West (NW) estimation, feasible generalized least squares (FGLS) estimation, and explicit modeling of an autoregressive structure of order one (AR(1)) in the error terms and (2) evaluating multiple imputation in the presence of data that are missing completely at random. FGLS and AR(1) outperformed NW and OLS estimation in terms of efficiency, Type I error rates, and coverage, while OLS was superior to the methods in terms of power for larger samples. The performance of all methods is consistent across 0% and 20% missing data conditions. 50% missing data led to unsatisfactory power and biased estimates. In light of these findings, we provide recommendations for applied researchers.

Causal Mediation Analysis in Single Case Experimental Designs: Introduction to the Special Issue.

This special issue of Evaluation and the Health Professions is dedicated to methods for causal mediation analysis in Single Case Experimental Designs (SCEDs). Mediation analysis is used to identify intermediate variables that transmit the effect of the independent variable on the outcome. Until recently, mediation analysis was mostly confined to between-subjects designs and panel studies with few exceptions. Consequently, most of the developments in causal mediation analysis have also been restricted to such designs. In applied health research, SCEDs have been used to evaluate total effects of treatments on outcomes of interest. Providing researchers with the methods for evaluating causal indirect effects for individual participants can lead to important improvements in diagnosis, treatment, and prevention. This special issue includes articles that describe advanced quantitative methods for testing mediators in SCEDs, propose and test approaches that allow for relaxing statistical assumptions that may not hold in real data, and illustrate mediation analysis for a single participant in real and simulated SCEDs data.

A Randomization Permutation Test for Single Subject Mediation.

In response to the importance of individual-level effects, the purpose of this paper is to describe the new randomization permutation (RP) test for a mediation mechanism for a single subject. We extend seminal work on permutation tests for individual-level data by proposing a test for mediation for one person. The method requires random assignment to the levels of the treatment variable at each measurement occasion, and repeated measures of the mediator and outcome from one subject. If several assumptions are met, the process by which a treatment changes an outcome can be statistically evaluated for a single subject, using the permutation mediation test method and the permutation confidence interval method for residuals. A simulation study evaluated the statistical properties of the new method suggesting that at least eight repeated measures are needed to control Type I error rates and larger sample sizes are needed for power approaching .8 even for large effects. The RP mediation test is a promising method for elucidating intraindividual processes of change that may inform personalized medicine and tailoring of process-based treatments for one subject.

A comparison of meta-methods for synthesizing indirect effects.

Synthesizing findings about the indirect (mediated) effect plays an important role in determining the mechanism through which variables affect one another. This simulation study compared six methods for synthesizing indirect effects: correlation-based MASEM, parameter-based MASEM, marginal likelihood synthesis, an adjustment to marginal likelihood synthesis, and univariate, and two-parameter sequential Bayesian methods. This paper provides an empirical example and code for using all methods compared in the simulation study. The methods were compared on (relative) bias, precision, and RMSE of the point estimates and the power, coverage, and type I error rates of the interval estimates. The factors in the simulation were the methods, the strength of the indirect effect, the measurement level of the independent variable, and the number of studies available for synthesis. Correlation-based MASEM had the lowest bias out of all methods and produced interval estimates with the best statistical properties. The precision of the point estimates and the RMSE was marginally different across methods. Marginal likelihood synthesis had the highest power but performed poorly in terms of coverage and type I error rates. The adjusted marginal likelihood synthesis and two-parameter sequential Bayesian methods performed adequately in terms of bias and power, and the adjusted marginal likelihood synthesis had higher power than the sequential Bayesian method. Correlation-based MASEM performed best out of the six methods. Guidelines for optimal practices when synthesizing indirect effects (eg, required number of studies, type of results reported) are provided, as well as suggestions for further methodological research.

A Tutorial in Bayesian Potential Outcomes Mediation Analysis.

Statistical mediation analysis is used to investigate intermediate variables in the relation between independent and dependent variables. Causal interpretation of mediation analyses is challenging because randomization of subjects to levels of the independent variable does not rule out the possibility of unmeasured confounders of the mediator to outcome relation. Furthermore, commonly used frequentist methods for mediation analysis compute the probability of the data given the null hypothesis, which is not the probability of a hypothesis given the data as in Bayesian analysis. Under certain assumptions, applying the potential outcomes framework to mediation analysis allows for the computation of causal effects, and statistical mediation in the Bayesian framework gives indirect effects probabilistic interpretations. This tutorial combines causal inference and Bayesian methods for mediation analysis so the indirect and direct effects have both causal and probabilistic interpretations. Steps in Bayesian causal mediation analysis are shown in the application to an empirical example.