Understated gender disparities due to outcome-dependent selection: Commentary on Mackelprang et al. (2023).

What is the gender gap in invited publications in high-impact psychology journals? To answer this critical question, Mackelprang et al. (2023) examined invited publications in five high-impact psychology journals. They first calculated the share of women among authors of the invited publications (35.6%), then compared it with a "base rate" (42.3%; the share of women among associate and full psychology professors at R1 institutions). This comparison was presented as empirical evidence of women being underrepresented in the authorship of publications in these high-impact journals. In this commentary, we show that comparing these two descriptives-either using a difference or a ratio-provides little insight into the actual gender disparity of interest. A fundamental shortcoming of such a comparison is due to outcome-dependent selection. We explain what outcome-dependent selection is and why it is inappropriate. Crucially, we explain why, following such outcome-dependent selection, comparing the share of women in the selected sample with a "base rate" rules out drawing valid inferences about the actual gender gap. We urge researchers to recognize the perils of, and thus avoid, outcome-dependent selection. Finally, we suggest an alternative approach that permits a more accurate understanding of gender disparities in academia. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Evaluating sensitivity to classification uncertainty in latent subgroup effect analyses.

BACKGROUND: Increasing attention is being given to assessing treatment effect heterogeneity among individuals belonging to qualitatively different latent subgroups. Inference routinely proceeds by first partitioning the individuals into subgroups, then estimating the subgroup-specific average treatment effects. However, because the subgroups are only latently associated with the observed variables, the actual individual subgroup memberships are rarely known with certainty in practice and thus have to be imputed. Ignoring the uncertainty in the imputed memberships precludes misclassification errors, potentially leading to biased results and incorrect conclusions. METHODS: We propose a strategy for assessing the sensitivity of inference to classification uncertainty when using such classify-analyze approaches for subgroup effect analyses. We exploit each individual's typically nonzero predictive or posterior subgroup membership probabilities to gauge the stability of the resultant subgroup-specific average causal effects estimates over different, carefully selected subsets of the individuals. Because the membership probabilities are subject to sampling variability, we propose Monte Carlo confidence intervals that explicitly acknowledge the imprecision in the estimated subgroup memberships via perturbations using a parametric bootstrap. The proposal is widely applicable and avoids stringent causal or structural assumptions that existing bias-adjustment or bias-correction methods rely on. RESULTS: Using two different publicly available real-world datasets, we illustrate how the proposed strategy supplements existing latent subgroup effect analyses to shed light on the potential impact of classification uncertainty on inference. First, individuals are partitioned into latent subgroups based on their medical and health history. Then within each fixed latent subgroup, the average treatment effect is assessed using an augmented inverse propensity score weighted estimator. Finally, utilizing the proposed sensitivity analysis reveals different subgroup-specific effects that are mostly insensitive to potential misclassification. CONCLUSIONS: Our proposed sensitivity analysis is straightforward to implement, provides both graphical and numerical summaries, and readily permits assessing the sensitivity of any machine learning-based causal effect estimator to classification uncertainty. We recommend making such sensitivity analyses more routine in latent subgroup effect analyses.

Estimating social influence in a social network using potential outcomes.

Social influence occurs when an individual's outcome is affected by another individual's actions. Current approaches in psychology that seek to examine social influence have focused on settings where individuals are nested in predefined groups and do not interact across groups. Such study designs permit using standard estimation methods such as multilevel models for estimating treatment effects but restrict social influence to originate only from individuals within the same group. In more general settings, such as social networks where an individual is free to interact with any other individual, the absence of discernible clusters or scientifically meaningful groups precludes existing estimation methods. In this article, we introduce a new class of methods for assessing social influence in social networks in the context of randomized experiments in psychology. Our proposal builds on the potential outcomes framework from the causal inference literature. In particular, we exploit the concept of (treatment) interference, which occurs between individuals when one individual's outcome is affected by other individuals' treatments. Estimation proceeds using randomization-based approaches that are established in other disciplines and guarantee valid inference by construction. We compared the proposed methods with standard methods empirically using Monte Carlo simulation studies. We illustrated the method using publicly available data from an experiment assessing the effects of an anticonflict intervention among students' peer networks. The R scripts used to implement the proposed methods in the simulation studies and the applied example are freely available online. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Nonlinear mediation analysis with high-dimensional mediators whose causal structure is unknown.

With multiple possible mediators on the causal pathway from a treatment to an outcome, we consider the problem of decomposing the effects along multiple possible causal path(s) through each distinct mediator. Under a path-specific effects framework, such fine-grained decompositions necessitate stringent assumptions, such as correctly specifying the causal structure among the mediators, and no unobserved confounding among the mediators. In contrast, interventional direct and indirect effects for multiple mediators can be identified under much weaker conditions, while providing scientifically relevant causal interpretations. Nonetheless, current estimation approaches require (correctly) specifying a model for the joint mediator distribution, which can be difficult when there is a high-dimensional set of possibly continuous and noncontinuous mediators. In this article, we avoid the need to model this distribution, by developing a definition of interventional effects previously suggested for longitudinal mediation. We propose a novel estimation strategy that uses nonparametric estimates of the (counterfactual) mediator distributions. Noncontinuous outcomes can be accommodated using nonlinear outcome models. Estimation proceeds via Monte Carlo integration. The procedure is illustrated using publicly available genomic data to assess the causal effect of a microRNA expression on the 3-month mortality of brain cancer patients that is potentially mediated by expression values of multiple genes.

Which behaviour change techniques are effective to promote physical activity and reduce sedentary behaviour in adults: a factorial randomized trial of an e- and m-health intervention.

BACKGROUND: E- and m-health interventions are promising to change health behaviour. Many of these interventions use a large variety of behaviour change techniques (BCTs), but it's not known which BCTs or which combination of BCTs contribute to their efficacy. Therefore, this experimental study investigated the efficacy of three BCTs (i.e. action planning, coping planning and self-monitoring) and their combinations on physical activity (PA) and sedentary behaviour (SB) against a background set of other BCTs. METHODS: In a 2 (action planning: present vs absent) × 2 (coping planning: present vs absent) × 2 (self-monitoring: present vs absent) factorial trial, 473 adults from the general population used the self-regulation based e- and m-health intervention 'MyPlan2.0' for five weeks. All combinations of BCTs were considered, resulting in eight groups. Participants selected their preferred target behaviour, either PA (n = 335, age = 35.8, 28.1% men) or SB (n = 138, age = 37.8, 37.7% men), and were then randomly allocated to the experimental groups. Levels of PA (MVPA in minutes/week) or SB (total sedentary time in hours/day) were assessed at baseline and post-intervention using self-reported questionnaires. Linear mixed-effect models were fitted to assess the impact of the different combinations of the BCTs on PA and SB. RESULTS: First, overall efficacy of each BCT was examined. The delivery of self-monitoring increased PA (t = 2.735, p = 0.007) and reduced SB (t = - 2.573, p = 0.012) compared with no delivery of self-monitoring. Also, the delivery of coping planning increased PA (t = 2.302, p = 0.022) compared with no delivery of coping planning. Second, we investigated to what extent adding BCTs increased efficacy. Using the combination of the three BCTs was most effective to increase PA (x2 = 8849, p = 0.003) whereas the combination of action planning and self-monitoring was most effective to decrease SB (x2 = 3.918, p = 0.048). To increase PA, action planning was always more effective in combination with coping planning (x2 = 5.590, p = 0.014; x2 = 17.722, p < 0.001; x2 = 4.552, p = 0.033) compared with using action planning without coping planning. Of note, the use of action planning alone reduced PA compared with using coping planning alone (x2 = 4.389, p = 0.031) and self-monitoring alone (x2 = 8.858, p = 003), respectively. CONCLUSIONS: This study provides indications that different (combinations of) BCTs may be effective to promote PA and reduce SB. More experimental research to investigate the effectiveness of BCTs is needed, which can contribute to improved design and more effective e- and m-health interventions in the future. TRIAL REGISTRATION: This study was preregistered as a clinical trial (ID number: NCT03274271 ). Release date: 20 October 2017.