Pooling methods for likelihood ratio tests in multiply imputed data sets.

Likelihood ratio tests (LRTs) are a popular tool for comparing statistical models. However, missing data are also common in empirical research, and multiple imputation (MI) is often used to deal with them. In multiply imputed data, there are multiple options for conducting LRTs, and new methods are still being proposed. In this article, we compare all available methods in multiple simulations covering applications in linear regression, generalized linear models, and structural equation modeling. In addition, we implemented these methods in an R package, and we illustrate its application in an example analysis concerned with the investigation of measurement invariance. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Unveiling mechanisms of change in digital interventions for depression: Study protocol for a systematic review and individual participant data meta-analysis.

Introduction: The efficacy and effectiveness of digital interventions for depression are both well-established. However, precise effect size estimates for mediators transmitting the effects of digital interventions are not available; and integrative insights on the specific mechanisms of change in internet- and mobile-based interventions (IMIs)-as related to key features like delivery type, accompanying support and theoretical foundation-are largely pending. Objective: We will conduct a systematic review and individual participant data meta-analysis (IPD-MA) evaluating the mediators associated with therapeutic change in various IMIs for depression in adults. Methods: We will use three electronic databases (i.e., Embase, Medline/PubMed, PsycINFO) as well as an already established database of IPD to identify relevant published and unpublished studies. We will include (1) randomized controlled trials that examine (2) mediators of (3) guided and unguided (4) IMIs with (5) various theoretical orientations for (6) adults with (7) clinically relevant symptoms of depression (8) compared to an active or passive control condition (9) with depression symptom severity as primary outcome. Study selection, data extraction, as well as quality and risk of bias (RoB) assessment will be done independently by two reviewers. Corresponding authors of eligible primary studies will be invited to share their IPD for this meta-analytic study. In a 1-stage IPD-MA, mediation analyses (e.g., on potential mediators like self-efficacy, emotion regulation or problem solving) will be performed using a multilevel structural equation modeling approach within a random-effects framework. Indirect effects will be estimated, with multiple imputation for missing data; the overall model fit will be evaluated and statistical heterogeneity will be assessed. Furthermore, we will investigate if indirect effects are moderated by different variables on participant- (e.g., age, sex/gender, symptom severity), study- (e.g., quality, studies evaluating the temporal ordering of changes in mediators and outcomes), and intervention-level (e.g., theoretical foundation, delivery type, guidance). Discussion: This systematic review and IPD-MA will generate comprehensive information on the differential strength of mediators and associated therapeutic processes in digital interventions for depression. The findings might contribute to the empirically-informed advancement of psychotherapeutic interventions, leading to more effective interventions and improved treatment outcomes in digital mental health. Besides, with our novel approach to mediation analyses with IPD-MA, we might also add to a methodological progression of evidence-synthesis in psychotherapy process research. Study registration with Open Science Framework OSF: https://osf.io/md7pq/.

Using synthetic data to improve the reproducibility of statistical results in psychological research.

In recent years, psychological research has faced a credibility crisis, and open data are often regarded as an important step toward a more reproducible psychological science. However, privacy concerns are among the main reasons that prevent data sharing. Synthetic data procedures, which are based on the multiple imputation (MI) approach to missing data, can be used to replace sensitive data with simulated values, which can be analyzed in place of the original data. One crucial requirement of this approach is that the synthesis model is correctly specified. In this article, we investigated the statistical properties of synthetic data with a particular emphasis on the reproducibility of statistical results. To this end, we compared conventional approaches to synthetic data based on MI with a data-augmented approach (DA-MI) that attempts to combine the advantages of masking methods and synthetic data, thus making the procedure more robust to misspecification. In multiple simulation studies, we found that the good properties of the MI approach strongly depend on the correct specification of the synthesis model, whereas the DA-MI approach can provide useful results even under various types of misspecification. This suggests that the DA-MI approach to synthetic data can provide an important tool that can be used to facilitate data sharing and improve reproducibility in psychological research. In a working example, we also demonstrate the implementation of these approaches in widely available software, and we provide recommendations for practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Response Surface Analysis with Missing Data.

Response Surface Analysis (RSA) is gaining popularity in psychological research as a tool for investigating congruence hypotheses (e.g., consequences of self-other agreement, person-job fit, dyadic similarity). RSA involves the estimation of a nonlinear polynomial regression model and the interpretation of the resulting response surface. However, little is known about how best to conduct RSA when the underlying data are incomplete. In this article, we compare different methods for handling missing data in RSA. This includes different strategies for multiple imputation (MI) and maximum-likelihood (ML) estimation. Specifically, we consider the "just another variable" (JAV) approach to MI and ML, an approach that is in regular use in applications of RSA, and the more novel "substantive-model-compatible" (SMC) approach. In a simulation study, we evaluate the impact of these methods on focal outcomes of RSA, including the accuracy of parameter estimates, the shape of the response surface, and the testing of congruence hypotheses. Our findings suggest that the JAV approach can sometimes distort parameter estimates and conclusions about the shape of the response surface, whereas the SMC approach performs well overall. We illustrate applications of the methods in a worked example with real data and provide recommendations for their application in practice.

Therapeutic processes in digital interventions for anxiety: A systematic review and meta-analytic structural equation modeling of randomized controlled trials.

While the efficacy of Internet- and mobile-based interventions (IMIs) for treating anxiety disorders is well established, there is no comprehensive overview about the underlying therapeutic processes so far. Thus, this systematic review and meta-analysis evaluated research on mediators and mechanisms of change in IMIs for adult anxiety disorders (PROSPERO: CRD42020185545). A systematic literature search was performed in five databases (i.e., CENTRAL, Embase, MEDLINE, PsycINFO and ClinicalTrials.gov). Two reviewers independently screened studies for inclusion, assessed the risk of bias and adherence to quality criteria for process research. Overall, 26 studies (N = 6042) investigating 64 mediators were included. Samples consisted predominantly of participants with clinically relevant symptoms of generalized anxiety disorder and severe health anxiety, as well as of participants with non-clinically relevant anxiety symptoms. The largest group of examined mediators (45%) were cognitive variables, evincing also the second highest proportion of significance (19/29); followed in numbers by skills (examined: 22%; significant: 10/14) and a wide range of other (19%; 7/12), emotional/affective (11%; 2/7) and behavioral mediators (3%; 1/2). Meta-analytical synthesis of mediators, limited by a small number of eligible studies, was conducted by deploying a two-stage structural equation modeling approach, resulting in a significant indirect effect for negative thinking (k = 3 studies) and non-significant indirect effects for combined cognitive variables, both in clinical (k = 5) and non-clinical samples (k = 3). The findings of this review might further the understanding on presumed change mechanisms in IMIs for anxiety, informing intervention development and the concurrent optimization of outcomes. Furthermore, by reviewing eligible mediation studies, we discuss methodological implications and recommendations for future process research, striving for causally robust findings. Future studies should investigate a broader range of variables as potential mediators, as well as to develop and apply original (digital) process and engagement measures to gather qualitative and high-resolution data on therapeutic processes.

Multiple imputation of missing data in multilevel models with the R package mdmb: a flexible sequential modeling approach.

Multilevel models often include nonlinear effects, such as random slopes or interaction effects. The estimation of these models can be difficult when the underlying variables contain missing data. Although several methods for handling missing data such as multiple imputation (MI) can be used with multilevel data, conventional methods for multilevel MI often do not properly take the nonlinear associations between the variables into account. In the present paper, we propose a sequential modeling approach based on Bayesian estimation techniques that can be used to handle missing data in a variety of multilevel models that involve nonlinear effects. The main idea of this approach is to decompose the joint distribution of the data into several parts that correspond to the outcome and explanatory variables in the intended analysis, thus generating imputations in a manner that is compatible with the substantive analysis model. In three simulation studies, we evaluate the sequential modeling approach and compare it with conventional as well as other substantive-model-compatible approaches to multilevel MI. We implemented the sequential modeling approach in the R package mdmb and provide a worked example to illustrate its application.

Efficacy and Moderators of Internet-Based Interventions in Adults with Subthreshold Depression: An Individual Participant Data Meta-Analysis of Randomized Controlled Trials.

INTRODUCTION: Evidence on effects of Internet-based interventions to treat subthreshold depression (sD) and prevent the onset of major depression (MDD) is inconsistent. OBJECTIVE: We conducted an individual participant data meta-analysis to determine differences between intervention and control groups (IG, CG) in depressive symptom severity (DSS), treatment response, close to symptom-free status, symptom deterioration and MDD onset as well as moderators of intervention outcomes. METHODS: Randomized controlled trials were identified through systematic searches via PubMed, PsycINFO, Embase and Cochrane Library. Multilevel regression analyses were used to examine efficacy and moderators. RESULTS: Seven trials (2,186 participants) were included. The IG was superior in DSS at all measurement points (posttreatment: 6-12 weeks; Hedges' g = 0.39 [95% CI: 0.25-0.53]; follow-up 1: 3-6 months; g = 0.30 [95% CI: 0.15-0.45]; follow-up 2: 12 months, g = 0.27 [95% CI: 0.07-0.47], compared with the CG. Significantly more participants in the IG than in the CG reached response and close to symptom-free status at all measurement points. A significant difference in symptom deterioration between the groups was found at the posttreatment assessment and follow-up 2. Incidence rates for MDD onset within 12 months were lower in the IG (19%) than in the CG (26%). Higher initial DSS and older age were identified as moderators of intervention effect on DSS. CONCLUSIONS: Our findings provide evidence for Internet-based interventions to be a suitable low-threshold intervention to treat individuals with sD and to reduce the incidence of MDD. This might be particularly true for older people with a substantial symptom burden.

Deep sedation versus general anaesthesia for transcatheter mitral valve repair: an individual patient data meta-analysis of observational studies.

AIMS: The aim of this meta-analysis was to compare general anaesthesia (GA) and deep sedation (DS) with regard to safety and length of intensive care unit (ICU) stay in patients undergoing percutaneous edge-to-edge mitral valve repair (PMVR). METHODS AND RESULTS: Four studies comparing GA and DS in patients undergoing PMVR were included in an individual patient data meta-analysis. Data were pooled after multiple imputation. The composite safety endpoint of all-cause death, stroke, pneumonia, or major to life-threatening bleeding occurred in 87 of 626 (13.9%) patients with no difference between patients treated with DS as compared to GA (56 and 31 events in 420 and 206 patients, respectively). In this regard, the odds ratio (OR) was 1.27 (95% confidence interval [CI]: 0.78 to 2.09; p=0.338) and 1.26 (95% CI: 0.49 to 3.22; p=0.496) following the one-stage and two-stage approach, respectively. Length of ICU stay was longer after GA as compared to DS (ratio of days 3.08, 95% CI: 2.18 to 4.36, p<0.001, and 2.88, 95% CI: 1.45 to 5.73, p=0.016, following the one-stage and two-stage approach, respectively). CONCLUSIONS: Both DS and GA might offer a similar safety profile. However, ICU stay seems to be shorter after DS.

Multiple imputation of missing data in multilevel designs: A comparison of different strategies.

Multiple imputation is a widely recommended means of addressing the problem of missing data in psychological research. An often-neglected requirement of this approach is that the imputation model used to generate the imputed values must be at least as general as the analysis model. For multilevel designs in which lower level units (e.g., students) are nested within higher level units (e.g., classrooms), this means that the multilevel structure must be taken into account in the imputation model. In the present article, we compare different strategies for multiply imputing incomplete multilevel data using mathematical derivations and computer simulations. We show that ignoring the multilevel structure in the imputation may lead to substantial negative bias in estimates of intraclass correlations as well as biased estimates of regression coefficients in multilevel models. We also demonstrate that an ad hoc strategy that includes dummy indicators in the imputation model to represent the multilevel structure may be problematic under certain conditions (e.g., small groups, low intraclass correlations). Imputation based on a multivariate linear mixed effects model was the only strategy to produce valid inferences under most of the conditions investigated in the simulation study. Data from an educational psychology research project are also used to illustrate the impact of the various multiple imputation strategies. (PsycINFO Database Record

Multiple imputation of missing covariate values in multilevel models with random slopes: a cautionary note.

Multiple imputation (MI) has become one of the main procedures used to treat missing data, but the guidelines from the methodological literature are not easily transferred to multilevel research. For models including random slopes, proper MI can be difficult, especially when the covariate values are partially missing. In the present article, we discuss applications of MI in multilevel random-coefficient models, theoretical challenges posed by slope variation, and the current limitations of standard MI software. Our findings from three simulation studies suggest that (a) MI is able to recover most parameters, but is currently not well suited to capture slope variation entirely when covariate values are missing; (b) MI offers reasonable estimates for most parameters, even in smaller samples or when its assumptions are not met; and