The Impact of Sparse Datasets When Harmonizing Data from Studies with Different Measures of the Same Construct.

Prevention science has increasingly turned to integrative data analysis (IDA) to combine individual participant-level data from multiple studies of the same topic, allowing us to evaluate overall effect size, test and model heterogeneity, and examine mediation. Studies included in IDA often use different measures for the same construct, leading to sparse datasets. We introduce a graph theory method for summarizing patterns of sparseness and use simulations to explore the impact of different patterns on measurement bias within three different measurement models: a single common factor, a hierarchical model, and a bifactor model. We simulated 1000 datasets with varying levels of sparseness and used Bayesian methods to estimate model parameters and evaluate bias. Results clarified that bias due to sparseness will depend on the strength of the general factor, the measurement model employed, and the level of indirect linkage among measures. We provide an example using a synthesis dataset that combined data on youth depression from 4146 youth who participated in 16 randomized field trials of prevention programs. Given that different synthesis datasets will embody different patterns of sparseness, we conclude by recommending that investigators use simulation methods to explore the potential for bias given the sparseness patterns they encounter.

Heterogeneity in the Effects of Interventions to Prevent Depression in Couples Facing Job Loss: Studying Baseline Target Moderation of Impact.

Couples' communication styles are associated with depression following job loss for both job seekers and their partners. The Couples Employment Program (CEP), an eight-session program for couples facing job loss, was developed to integrate job search strategies sessions from the JOBS program with couple communication sessions, targeting job search behavior, motivation, mastery, and couple communication. We hypothesized that CEP would have compensatory effects, such that those who began the program with lower job search behavior, lower motivation, less mastery, and more negative or less positive couple communication would make more gains on these targets, and this would mediate impact on reducing risk for depression. We conducted a randomized field trial of CEP with 1477 heterosexual couples facing recent unemployment. Baseline levels of job search behavior and motivation, but not mastery or depression, moderated the impact of intervention on job seeker depression slopes over 12 months; job seekers reporting less job search behavior and motivation at baseline benefited more. Male partners with higher baseline depression also benefited. Opposite to our hypothesis, baseline levels of couple's communication moderated the impact of intervention such that partners in couples with more negative and less positive communication showed iatrogenic effects. There was no evidence that baseline target levels moderated the impact of the intervention on any of these targets. We speculate that more intensive communication training may be necessary for positive impact.

When Is It Time to Revise or Adapt Our Prevention Programs? Introduction to Special Issue on Using Baseline Target Moderation to Assess Variation in Prevention Impact.

Adaptation of preventive interventions to increase their impact can be advanced by identifying subgroups or specific contexts where program effects appear stronger or weaker. But how do we know where to look for effect heterogeneity in ways that will inform successful adaptation? This paper introduces a special issue that brings together research across prevention science sub-disciplines that adopted baseline target moderated mediation (BTMM) designs to search for effect heterogeneity and guide adaptation of established prevention programs. For this special issue intervention  scientists analyzed data from randomized trials using BTM and BTMM models, evaluating evidence for variation in intervention impact for trials spanning different health outcomes, different developmental periods, and different social units. This introduction provides a brief summary of the various patterns of effect reported in these papers, noting that the most common pattern involved compensatory effects (those beginning the trial with greater risk or fewer protective factors benefit the most), but other patterns including rich-get-richer and partially iatrogenic effects were also detected. This paper ends with a discussion of methodological and substantive implications of these findings for future prevention research, including next-generation prevention trials.

Retrospective Psychometrics and Effect Heterogeneity in Integrated Data Analysis: Commentary on the Special Issue.

The current special issue of Prevention Science indicates that momentum in using individual participant data (IPD) and integrative data analysis (IDA) to combine and synthesize findings in prevention science has accelerated over the past decade. In this commentary, we focus on two general themes involving methods for harmonizing measures and findings of effect heterogeneity. We describe methods for harmonization as retrospective psychometrics, requiring that we attend to the assumptions necessary for accurate measurement, but adjust our methods given the constraints of working with existing datasets that often involve different measures in different studies. We point to novel approaches for increasing confidence that semantic matching and empirical modeling used in these studies will yield accurate and valid measurements that can be combined in IDA. We also review findings about effect heterogeneity, emphasizing the importance of using etiologic and action theories to identify and evaluate sources of such effects. We note that all of the papers in this issue deserve careful attention, as they illustrate how prevention scientists are approaching the complexities of IDA and exploring novel methods for overcoming its challenges.

Familias Unidas Prevents Youth Internalizing Symptoms: a Baseline Target Moderated Mediation (BTMM) Study.

Family-based preventive interventions have been found to prevent youth internalizing symptoms, yet they operate through diverse mechanisms with heterogeneous effects for different youth. To better target preventive interventions, this study examines the effects of the Familias Unidas preventive intervention on reducing internalizing symptoms with a universal sample of Hispanic youth in a real-world school setting (i.e., effectiveness trial). The study utilizes emerging methods in baseline target moderated mediation (BTMM) to determine whether the intervention reduces internalizing symptoms through its impact on three distinct mechanisms: family functioning, parent stress, and social support for parents. Data are from a randomized controlled effectiveness trial of 746 Hispanic eighth graders and their parents assessed at baseline, 6-, 18-, and 30-month post-baseline. BTMM models examined three moderated mechanisms through which the intervention might influence 30-month adolescent internalizing symptoms. The intervention decreased youth internalizing symptoms through improvements in family functioning in some models, but there was no evidence of moderation by baseline level of family functioning. There was some evidence of mediation through increasing social support for parents for those intervention parents presenting with lower baseline support. However, there was no evidence of mediation through parent stress. Post hoc analyses suggest a possible cascading of effects where improvements in support for parents strengthened parental monitoring of youth and ultimately reduced youth internalizing symptoms. Findings support the intervention's effects on internalizing symptoms in a universal, real-world setting, and the value of BTMM methods to improve the targeting of preventive interventions. ClinicalTrials.gov Identifier: NCTO1038206, First Posted: December 23, 2009.

Re-envisioning, Retooling, and Rebuilding Prevention Science Methods to Address Structural and Systemic Racism and Promote Health Equity.

The historic momentum from national conversations on the roots and current impacts of racism in the USA presents an incredible window of opportunity for prevention scientists to revisit how common theories, measurement tools, methodologies, and interventions can be radically re-envisioned, retooled, and rebuilt to dismantle racism and promote equitable health for minoritized communities. Recognizing this opportunity, the NIH-funded Prevention Science and Methodology Group (PSMG) launched a series of presentations focused on the role of Prevention Science to address racism and discrimination guided by a commitment to social justice and health equity. The current manuscript aims to advance the field of Prevention Science by summarizing key issues raised during the series' presentations and proposing concrete research priorities and steps that hold promise for promoting health equity by addressing systemic racism. Being anti-racist is an active practice for all of us, whether we identify as methodologists, interventionists, practitioners, funders, community members, or an intersection of these identities. We implore prevention scientists and methodologists to take on these conversations with us to promote science and practice that offers every life the right to live in a just and equitable world.

Unemployed Individuals Reporting Hindrance Work Stress at Previous Job Have Increased Likelihood of Insomnia Disorder.

BACKGROUND: Workplace stress and unemployment are each associated with disturbances in sleep. However, a substantial gap exists in what we know about the type of workplace stress preceding job loss and the lasting effect workplace stressors may have on long-term health outcomes. We hypothesized that a specific type of workplace stress, hindrance stress, would be a stronger predictor of current insomnia disorder, compared to challenge stress. METHOD: Cross-sectional data were analyzed from 191 recently unemployed individuals participating in the ongoing Assessing Daily Patterns through occupational Transitions (ADAPT) study. Participants were administered the Cavanaugh et al. (J Appl Psychol. 85(1):65, 2000) self-reported work stress scale regarding their previous job and the Duke Sleep Interview (DSI-SD), a semi-structured interview assessing ICSD-3 insomnia disorder (chronic and acute). RESULTS: Results from logistic regression analyses indicated that hindrance work stress was associated with an increased likelihood of current overall, chronic, and acute insomnia disorder, when controlling for challenge stress and significant demographic factors. Challenge stress was associated with an increased likelihood of chronic insomnia disorder when controlling for hindrance stress and covariates. The association between challenge stress and acute insomnia differed as a function of sex. CONCLUSION: Hindrance work stressors were associated with increased odds of current insomnia disorder, even after employment ended. Across each of the tested models, hindrance stress had stronger effects on insomnia than challenge stress. These findings support and extend both the challenge-hindrance framework of work-related stress and the 3 P model of insomnia.

Using baseline target moderation to guide decisions on adapting prevention programs.

Tom Dishion, a pioneer in prevention science, was one of the first to recognize the importance of adapting interventions to the needs of individual families. Building towards this goal, we suggest that prevention trials be used to assess baseline target moderated mediation (BTMM), where preventive intervention effects are mediated through change in specific targets, and the resulting effect varies across baseline levels of the target. Four forms of BTMM found in recent trials are discussed including compensatory, rich-get-richer, crossover, and differential iatrogenic effects. A strategy for evaluating meaningful preventive effects is presented based on preventive thresholds for diagnostic conditions, midpoint targets and proximal risk or protective mechanisms. Methods are described for using the results from BTMM analyses of these thresholds to estimate indices of intervention risk reduction or increase as they vary over baseline target levels, and potential cut points are presented for identifying subgroups that would benefit from program adaptation because of weak or potentially iatrogenic program effects. Simulated data are used to illustrate curves for the four forms of BTMM effects and how implications for adaptation change when untreated control group outcomes also vary over baseline target levels.

Preventive Effect Heterogeneity: Causal Inference in Personalized Prevention.

This paper employs a causal inference framework to explore two logically distinct forms of preventive effect heterogeneity relevant for studying variation in preventive effect as a basis for developing more personalized interventions. Following VanderWeele (2015), I begin with a discussion of causal interaction involving manipulable moderators that combine to yield more complex nonadditive effects. This is contrasted with effect heterogeneity, which involves variation in causal structure indexed by stable characteristics of populations or contexts. The paper then discusses one particularly promising approach, the baseline target moderated mediation (BTMM) design, which uses theoretically informed baseline target moderators to strengthen causal inference, suggesting methods for using BTMM designs to develop targeting strategies for personalized prevention. It presents examples of recent intervention trials that apply these different forms of moderation, and discusses causal inference and the problem of moderation confounding, reviewing methods for minimizing its impact, including recent advances in the use of propensity score matching.

Programs for Preventing Depression in Adolescence: Who Benefits and Who Does Not? An Introduction to the Supplemental Issue.

We introduce this supplemental issue of Prevention Science, which brings together a set of papers from leading investigators who have conducted trials testing whether intervention programs prevent adolescent depression. Using data from these trials, these papers explore a series of factors that might account for variation in intervention benefit, employing several novel methods for assessing effect heterogeneity. These studies follow two general paradigms: three papers report findings from single randomized preventive intervention trials, while the remaining papers develop and apply new methods for combining data from multiple studies to evaluate effect heterogeneity more broadly. Colleagues from NIMH and SAMHSA also provide commentaries on these studies. They conclude that synthesis of findings from multiple trials holds great promise for advancing the field, and progress will be accelerated if collaborative data sharing becomes the norm rather than the exception.

Limitations in Using Multiple Imputation to Harmonize Individual Participant Data for Meta-Analysis.

Individual participant data (IPD) meta-analysis is a meta-analysis in which the individual-level data for each study are obtained and used for synthesis. A common challenge in IPD meta-analysis is when variables of interest are measured differently in different studies. The term harmonization has been coined to describe the procedure of placing variables on the same scale in order to permit pooling of data from a large number of studies. Using data from an IPD meta-analysis of 19 adolescent depression trials, we describe a multiple imputation approach for harmonizing 10 depression measures across the 19 trials by treating those depression measures that were not used in a study as missing data. We then apply diagnostics to address the fit of our imputation model. Even after reducing the scale of our application, we were still unable to produce accurate imputations of the missing values. We describe those features of the data that made it difficult to harmonize the depression measures and provide some guidelines for using multiple imputation for harmonization in IPD meta-analysis.

Preventing Youth Internalizing Symptoms Through the Familias Unidas Intervention: Examining Variation in Response.

Prevention programs that strengthen parenting and family functioning have been found to reduce poor behavioral outcomes in adolescents, including substance use, HIV risk, externalizing and internalizing problems. However, there is evidence that not all youth benefit similarly from these programs. Familias Unidas is a family-focused intervention designed to prevent substance use and sexual risk among Hispanic youth and has recently demonstrated unanticipated reductions in internalizing symptoms for some youth. This paper examines variation in intervention response for internalizing symptoms using individual-level data pooled across four distinct Familias Unidas trials: (1) 266 eighth grade students recruited from the general school population; (2) 160 ninth grade students from the general school population; (3) 213 adolescents with conduct, aggression, and/or attention problems; and (4) 242 adolescents with a delinquency history. Causal inference growth mixture modeling suggests a three-class model. The two largest classes represent youth with low (60 %) and medium (27 %) internalizing symptoms at baseline, and both intervention and control participants show reductions in internalizing symptoms. The third class (13 %) represents youth with high levels of baseline internalizing symptoms who remain at steady levels of internalizing symptoms when exposed to the intervention, but who experience an increase in symptoms under the control condition. Female gender, low baseline levels of parent-adolescent communication, and older age were associated with membership in the high-risk class. These synthesis analyses involving a large sample of youth with varying initial risk levels represent a further step toward strengthening our knowledge of preventive intervention response and improving preventive interventions.

Addressing Methodologic Challenges and Minimizing Threats to Validity in Synthesizing Findings from Individual-Level Data Across Longitudinal Randomized Trials.

Integrative Data Analysis (IDA) encompasses a collection of methods for data synthesis that pools participant-level data across multiple studies. Compared with single-study analyses, IDA provides larger sample sizes, better representation of participant characteristics, and often increased statistical power. Many of the methods currently available for IDA have focused on examining developmental changes using longitudinal observational studies employing different measures across time and study. However, IDA can also be useful in synthesizing across multiple randomized clinical trials to improve our understanding of the comprehensive effectiveness of interventions, as well as mediators and moderators of those effects. The pooling of data from randomized clinical trials presents a number of methodological challenges, and we discuss ways to examine potential threats to internal and external validity. Using as an illustration a synthesis of 19 randomized clinical trials on the prevention of adolescent depression, we articulate IDA methods that can be used to minimize threats to internal validity, including (1) heterogeneity in the outcome measures across trials, (2) heterogeneity in the follow-up assessments across trials, (3) heterogeneity in the sample characteristics across trials, (4) heterogeneity in the comparison conditions across trials, and (5) heterogeneity in the impact trajectories. We also demonstrate a technique for minimizing threats to external validity in synthesis analysis that may result from non-availability of some trial datasets. The proposed methods rely heavily on latent variable modeling extensions of the latent growth curve model, as well as missing data procedures. The goal is to provide strategies for researchers considering IDA.

Two-Year Impact of Prevention Programs on Adolescent Depression: an Integrative Data Analysis Approach.

This paper presents the first findings of an integrative data analysis of individual-level data from 19 adolescent depression prevention trials (n = 5210) involving nine distinct interventions across 2 years post-randomization. In separate papers, several interventions have been found to decrease the risk of depressive disorders or elevated depressive/internalizing symptoms among youth. One type of intervention specifically targets youth without a depressive disorder who are at risk due to elevated depressive symptoms and/or having a parent with a depressive disorder. A second type of intervention targets two broad domains: prevention of problem behaviors, which we define as drug use/abuse, sexual risk behaviors, conduct disorder, or other externalizing problems, and general mental health. Most of these latter interventions improve parenting or family factors. We examined the shared and unique effects of these interventions by level of baseline youth depressive symptoms, sociodemographic characteristics of the youth (age, sex, parent education, and family income), type of intervention, and mode of intervention delivery to the youth, parent(s), or both. We harmonized eight different measures of depression utilized across these trials and used growth models to evaluate intervention impact over 2 years. We found a significant overall effect of these interventions on reducing depressive symptoms over 2 years and a stronger impact among those interventions that targeted depression specifically rather than problem behaviors or general mental health, especially when baseline symptoms were high. Implications for improving population-level impact are discussed.

Longitudinal assessment of daily activity patterns on weight change after involuntary job loss: the ADAPT study protocol.

BACKGROUND: The World Health Organization has identified obesity as one of the most visible and neglected public health problems worldwide. Meta-analytic studies suggest that insufficient sleep increases the risk of developing obesity and related serious medical conditions. Unfortunately, the nationwide average sleep duration has steadily declined over the last two decades with 25% of U.S. adults reporting insufficient sleep. Stress is also an important indirect factor in obesity, and chronic stress and laboratory-induced stress negatively impact sleep. Despite what we know from basic sciences about (a) stress and sleep and (b) sleep and obesity, we know very little about how these factors actually manifest in a natural environment. The Assessing Daily Activity Patterns Through Occupational Transitions (ADAPT) study tests whether sleep disruption plays a key role in the development of obesity for individuals exposed to involuntary job loss, a life event that is often stressful and disrupting to an individual's daily routine. METHODS: This is an 18-month closed, cohort research design examining social rhythms, sleep, dietary intake, energy expenditure, waist circumference, and weight gain over 18 months in individuals who have sustained involuntary job loss. Approximately 332 participants who lost their job within the last 3 months are recruited from flyers within the Arizona Department of Economic Security (AZDES) Unemployment Insurance Administration application packets and other related postings. Multivariate growth curve modeling will be used to investigate the temporal precedence of changes in social rhythms, sleep, and weight gain. DISCUSSION: It is hypothesized that: (1) unemployed individuals with less consistent social rhythms and worse sleep will have steeper weight gain trajectories over 18 months than unemployed individuals with stable social rhythms and better sleep; (2) disrupted sleep will mediate the relationship between social rhythm disruption and weight gain; and (3) reemployment will be associated with a reversal in the negative trajectories outlined above. Positive findings will provide support for the development of obesity prevention campaigns targeting sleep and social rhythms in an accessible subgroup of vulnerable individuals.

Combining Stress Exposure and Stress Generation: Does Neuroticism Alter the Dynamic Interplay of Stress, Depression, and Anxiety Following Job Loss?

OBJECTIVE: Emerging models of stress point to a dynamic formulation where stressors and internalizing symptoms reciprocally influence each other. This study tested whether this dynamic interplay is the result of a general internalizing process underlying both depression and anxiety, and whether it varies with neuroticism. METHOD: A total of 426 adults (51% female; 47% White, 42% African American) were assessed five times over 6 months following loss of employment, using repeated measurements of stressors, depression, anxiety, and neuroticism. RESULTS: Latent growth across 6 months and multilevel cross-lagged regressions across 6 weeks supported the hypothesis that stressors and internalizing symptoms have reciprocal effects after job loss. Findings for unique variation in depression paralleled those for general internalizing, whereas few findings emerged for general or social anxiety after controlling for internalizing. Neuroticism strengthened the association of change in stressors with change in symptoms across 6 months. Those with high neuroticism showed less reduction in internalizing following reemployment and were less likely to be reemployed when starting with higher internalizing. CONCLUSIONS: The moderated reciprocal effects model helps account for onset, maintenance, and resolution of symptoms following job loss. We speculate that these findings may be due in part to differential emotion regulation and reductions in motivation.

Testing moderation in network meta-analysis with individual participant data.

Meta-analytic methods for combining data from multiple intervention trials are commonly used to estimate the effectiveness of an intervention. They can also be extended to study comparative effectiveness, testing which of several alternative interventions is expected to have the strongest effect. This often requires network meta-analysis (NMA), which combines trials involving direct comparison of two interventions within the same trial and indirect comparisons across trials. In this paper, we extend existing network methods for main effects to examining moderator effects, allowing for tests of whether intervention effects vary for different populations or when employed in different contexts. In addition, we study how the use of individual participant data may increase the sensitivity of NMA for detecting moderator effects, as compared with aggregate data NMA that employs study-level effect sizes in a meta-regression framework. A new NMA diagram is proposed. We also develop a generalized multilevel model for NMA that takes into account within-trial and between-trial heterogeneity and can include participant-level covariates. Within this framework, we present definitions of homogeneity and consistency across trials. A simulation study based on this model is used to assess effects on power to detect both main and moderator effects. Results show that power to detect moderation is substantially greater when applied to individual participant data as compared with study-level effects. We illustrate the use of this method by applying it to data from a classroom-based randomized study that involved two sub-trials, each comparing interventions that were contrasted with separate control groups. Copyright © 2016 John Wiley & Sons, Ltd.

Reducing Internalizing Symptoms Among High-Risk, Hispanic Adolescents: Mediators of a Preventive Family Intervention.

Familias Unidas is a family-focused preventive intervention that has been found to reduce drug use and sexual risk behaviors among Hispanic adolescents. In some trials, Familias Unidas has also been found to be efficacious in reducing adolescent internalizing symptoms (i.e., depressive and anxiety symptoms), even though the intervention did not specifically target internalizing symptoms. This study examines potential mediators or mechanisms by which Familias Unidas influences internalizing symptoms, specifically the role of intervention-targeted improvements in parent-adolescent communication and reductions in youth externalizing behaviors. A total of 213 Hispanic eighth grade students with a history of externalizing behavior problems and their primary caregivers were recruited from the public school system. Participants, with a mean age of 13.8 years, were randomized into the Familias Unidas intervention or community practice control condition and assessed at baseline, 6, 18, and 30 months post-baseline. A cascading mediation model was tested in which the Familias Unidas intervention was hypothesized to decrease adolescent internalizing symptoms through two mediators: improvements in parent-adolescent communication leading to decreases in externalizing behaviors. Findings show that the intervention had significant direct effects on youth internalizing symptoms at 30 months post-baseline. In addition, the cascading mediation model was supported in which the Familias Unidas intervention predicted significant improvements in parent-adolescent communication at 6 months, subsequently decreasing externalizing behaviors at 18 months, and ultimately reducing youth internalizing symptoms at 30 months post-baseline. Implications for prevention interventions are discussed.

Human Subjects Protection and Technology in Prevention Science: Selected Opportunities and Challenges.

Internet-connected devices are changing the way people live, work, and relate to one another. For prevention scientists, technological advances create opportunities to promote the welfare of human subjects and society. The challenge is to obtain the benefits while minimizing risks. In this article, we use the guiding principles for ethical human subjects research and proposed changes to the Common Rule regulations, as a basis for discussing selected opportunities and challenges that new technologies present for prevention science. The benefits of conducting research with new populations, and at new levels of integration into participants' daily lives, are presented along with five challenges along with technological and other solutions to strengthen the protections that we provide: (1) achieving adequate informed consent with procedures that are acceptable to participants in a digital age; (2) balancing opportunities for rapid development and broad reach, with gaining adequate understanding of population needs; (3) integrating data collection and intervention into participants' lives while minimizing intrusiveness and fatigue; (4) setting appropriate expectations for responding to safety and suicide concerns; and (5) safeguarding newly available streams of sensitive data. Our goal is to promote collaboration between prevention scientists, institutional review boards, and community members to safely and ethically harness advancing technologies to strengthen impact of prevention science.