Background: Single-session interventions have the potential to address young people's mental health needs at scale, but their effects are heterogeneous. We tested whether the mindset + supportive context hypothesis could help explain when intervention effects persist or fade over time. The hypothesis posits that interventions are more effective in environments that support the intervention message. We tested this hypothesis using the synergistic mindsets intervention, a preventative treatment for stress-related mental health symptoms that helps students appraise stress as a potential asset in the classroom (e.g., increasing oxygenated blood flow) rather than debilitating. In an introductory college course, we examined whether intervention-consistent messages from instructors sustained changes in appraisals over time, as well as impacts on students' predisposition to try demanding academic tasks that could enhance learning. Methods: We randomly assigned 1675 students in the course to receive the synergistic mindsets intervention (or a control activity) at the beginning of the semester, and subsequently, to receive intervention-supportive messages from their instructor (or neutral messages) four times throughout the term. We collected weekly measures of students' appraisals of stress in the course and their predisposition to take on academic challenges. Trial-registration: OSF.io; DOI: 10.17605/osf.io/fchyn. Results: A conservative Bayesian analysis indicated that receiving both the intervention and supportive messages led to the greatest increases in positive stress appraisals (0.35 SD; 1.00 posterior probability) and challenge-seeking predisposition (2.33 percentage points; 0.94 posterior probability), averaged over the course of the semester. In addition, intervention effects grew larger throughout the semester when complemented by supportive instructor messages, whereas without these messages, intervention effects shrank somewhat over time. Conclusions: This study shows, for the first time, that supportive cues in local contexts can be the difference in whether a single-session intervention's effects fade over time or persist and even amplify.
Meta-analysts often ask a yes-or-no question: Is there an intervention effect or not? This traditional, all-or-nothing thinking stands in contrast with current best practice in meta-analysis, which calls for a heterogeneity-attuned approach (i.e., focused on the extent to which effects vary across procedures, participant groups, or contexts). This heterogeneity-attuned approach allows researchers to understand where effects are weaker or stronger and reveals mechanisms. The current article builds on a rare opportunity to compare two recent meta-analyses that examined the same literature (growth mindset interventions) but used different methods and reached different conclusions. One meta-analysis used a traditional approach (Macnamara and Burgoyne, in press), which aggregated effect sizes for each study before combining them and examined moderators one-by-one by splitting the data into small subgroups. The second meta-analysis (Burnette et al., in press) modeled the variation of effects within studies-across subgroups and outcomes-and applied modern, multi-level meta-regression methods. The former concluded that growth mindset effects are biased, but the latter yielded nuanced conclusions consistent with theoretical predictions. We explain why the practices followed by the latter meta-analysis were more in line with best practices for analyzing large and heterogeneous literatures. Further, an exploratory re-analysis of the data showed that applying the modern, heterogeneity-attuned methods from Burnette et al. (in press) to the dataset employed by Macnamara and Burgoyne (in press) confirmed Burnette et al.'s conclusions; namely, that there was a meaningful, significant effect of growth mindset in focal (at-risk) groups. This article concludes that heterogeneity-attuned meta-analysis is important both for advancing theory and for avoiding the boom-or-bust cycle that plagues too much of psychological science.
When do adolescents' dreams of promising journeys through high school translate into academic success? This monograph reports the results of a collaborative effort among sociologists and psychologists to systematically examine the role of schools and classrooms in disrupting or facilitating the link between adolescents' expectations for success in math and their subsequent progress in the early high school math curriculum. Our primary focus was on gendered patterns of socioeconomic inequality in math and how they are tethered to the school's peer culture and to students' perceptions of gender stereotyping in the classroom. To do this, this monograph advances Mindset × Context Theory. This orients research on educational equity to the reciprocal influence between students' psychological motivations and their school-based opportunities to enact those motivations. Mindset × Context Theory predicts that a student's mindset will be more strongly linked to developmental outcomes among groups of students who are at risk for poor outcomes, but only in a school or classroom context where there is sufficient need and support for the mindset. Our application of this theory centers on expectations for success in high school math as a foundational belief for students' math progress early in high school. We examine how this mindset varies across interpersonal and cultural dynamics in schools and classrooms. Following this perspective, we ask: 1. Which gender and socioeconomic identity groups showed the weakest or strongest links between expectations for success in math and progress through the math curriculum? 2. How did the school's peer culture shape the links between student expectations for success in math and math progress across gender and socioeconomic identity groups? 3. How did perceptions of classroom gender stereotyping shape the links between student expectations for success in math and math progress across gender and socioeconomic identity groups? We used nationally representative data from about 10,000 U.S. public school 9th graders in the National Study of Learning Mindsets (NSLM) collected in 2015-2016-the most recent, national, longitudinal study of adolescents' mindsets in U.S. public schools. The sample was representative with respect to a large number of observable characteristics, such as gender, race, ethnicity, English Language Learners (ELLs), free or reduced price lunch, poverty, food stamps, neighborhood income and labor market participation, and school curricular opportunities. This allowed for generalization to the U.S. public school population and for the systematic investigation of school- and classroom-level contextual factors. The NSLM's complete sampling of students within schools also allowed for a comparison of students from different gender and socioeconomic groups with the same expectations in the same educational contexts. To analyze these data, we used the Bayesian Causal Forest (BCF) algorithm, a best-in-class machine-learning method for discovering complex, replicable interaction effects. Chapter IV examined the interplay of expectations, gender, and socioeconomic status (SES; operationalized with maternal educational attainment). Adolescents' expectations for success in math were meaningful predictors of their early math progress, even when controlling for other psychological factors, prior achievement in math, and racial and ethnic identities. Boys from low-SES families were the most vulnerable identity group. They were over three times more likely to not make adequate progress in math from 9th to 10th grade relative to girls from high-SES families. Boys from low-SES families also benefited the most from their expectations for success in math. Overall, these results were consistent with Mindset × Context Theory's predictions. Chapters V and VI examined the moderating role of school-level and classroom-level factors in the patterns reported in Chapter IV. Expectations were least predictive of math progress in the highest-achieving schools and schools with the most academically oriented peer norms, that is, schools with the most formal and informal resources. School resources appeared to compensate for lower levels of expectations. Conversely, expectations most strongly predicted math progress in the low/medium-achieving schools with less academically oriented peers, especially for boys from low-SES families. This chapter aligns with aspects of Mindset × Context Theory. A context that was not already optimally supporting student success was where outcomes for vulnerable students depended the most on student expectations. Finally, perceptions of classroom stereotyping mattered. Perceptions of gender stereotyping predicted less progress in math, but expectations for success in math more strongly predicted progress in classrooms with high perceived stereotyping. Gender stereotyping interactions emerged for all sociodemographic groups except for boys from high-SES families. The findings across these three analytical chapters demonstrate the value of integrating psychological and sociological perspectives to capture multiple levels of schooling. It also drew on the contextual variability afforded by representative sampling and explored the interplay of lab-tested psychological processes (expectations) with field-developed levers of policy intervention (school contexts). This monograph also leverages developmental and ecological insights to identify which groups of students might profit from different efforts to improve educational equity, such as interventions to increase expectations for success in math, or school programs that improve the school or classroom cultures.
Academic Success , Mathematics , Motivation , Adolescent , Female , Humans , Male , Bayes Theorem , Longitudinal Studies , Schools
Educational outcomes remain highly unequal within and across nations. Students' mindsets-their beliefs about whether intellectual abilities can be developed-have been identified as a potential lever for making adolescents' academic outcomes more equitable. Recent research, however, suggests that intervention programs aimed at changing students' mindsets should be supplemented by programs aimed at the changing the mindset culture, which is defined as the shared set of beliefs about learning in a school or classroom. This paper reviews the theoretical and empirical origin of the mindset culture and examines its potential to reduce group-based inequalities in education. In particular, experiments have identified two broad ways the mindset culture is communicated by teachers: via informal messages about growth (e.g., that all students will be helped to learn and succeed), and formal opportunities to improve (e.g., learning-focused grading policies and opportunities to revise and earn credit). New field experiments, applying techniques from behavioral science, have also revealed effective ways to influence teachers' culture-creating behaviors. This paper describes recent breakthroughs in the U.S. educational context and discusses how lessons from these studies might be applied in future, global collaborations with researchers and practitioners.
Chater & Loewenstein (C&L) ignore the long history by which social scientists have developed more nuanced and ultimately more helpful ways to understand the relationship between persons and situations. This tradition is reflected and advanced in a large literature on "wise" social-psychological or mindset interventions, which C&L do not discuss yet mischaracterize.
Group-based educational disparities are smaller in classrooms where teachers express a belief that students can improve their abilities. However, a scalable method for motivating teachers to adopt such growth mindset-supportive teaching practices has remained elusive. In part, this is because teachers often already face overwhelming demands on their time and attention and have reason to be skeptical of the professional development advice they receive from researchers and other experts. We designed an intervention that overcame these obstacles and successfully motivated high-school teachers to adopt specific practices that support students' growth mindsets. The intervention used the values-alignment approach. This approach motivates behavioral change by framing a desired behavior as aligned with a core value-one that is an important criterion for status and admiration in the relevant social reference group. First, using qualitative interviews and a nationally representative survey of teachers, we identified a relevant core value: inspiring students' enthusiastic engagement with learning. Next, we designed a ~45-min, self-administered, online intervention that persuaded teachers to view growth mindset-supportive practices as a way to foster such student engagement and thus live up to that value. We randomly assigned 155 teachers (5,393 students) to receive the intervention and 164 teachers (6,167 students) to receive a control module. The growth mindset-supportive teaching intervention successfully promoted teachers' adoption of the suggested practices, overcoming major barriers to changing teachers' classroom practices that other scalable approaches have failed to surmount. The intervention also substantially improved student achievement in socioeconomically disadvantaged classes, reducing inequality in educational outcomes.
Academic Success , Internet-Based Intervention , Humans , Educational Status , Students , Learning
A promising way to mitigate inequality is by addressing students' worries about belonging. But where and with whom is this social-belonging intervention effective? Here we report a team-science randomized controlled experiment with 26,911 students at 22 diverse institutions. Results showed that the social-belonging intervention, administered online before college (in under 30 minutes), increased the rate at which students completed the first year as full-time students, especially among students in groups that had historically progressed at lower rates. The college context also mattered: The intervention was effective only when students' groups were afforded opportunities to belong. This study develops methods for understanding how student identities and contexts interact with interventions. It also shows that a low-cost, scalable intervention generalizes its effects to 749 4-year institutions in the United States.
Achievement , Social Identification , Students , Humans , Students/psychology , Universities , Random Allocation , Psychosocial Intervention
Psychological well-being, characterized by feelings, cognitions, and strategies that are associated with positive functioning (including hedonic and eudaimonic well-being), has been linked with better physical health and greater longevity. Importantly, psychological well-being can be strengthened with interventions, providing a strategy for improving population health. But are the effects of well-being interventions meaningful, durable, and scalable enough to improve health at a population-level? To assess this possibility, a cross-disciplinary group of scholars convened to review current knowledge and develop a research agenda. Here we summarize and build on the key insights from this convening, which were: (1) existing interventions should continue to be adapted to achieve a large-enough effect to result in downstream improvements in psychological functioning and health, (2) research should determine the durability of interventions needed to drive population-level and lasting changes, (3) a shift from individual-level care and treatment to a public-health model of population-level prevention is needed and will require new infrastructure that can deliver interventions at scale, (4) interventions should be accessible and effective in racially, ethnically, and geographically diverse samples. A discussion examining the key future research questions follows.
Behavioral science interventions have the potential to address longstanding policy problems, but their effects are typically heterogeneous across contexts (e.g., teachers, schools, and geographic regions). This contextual heterogeneity is poorly understood, however, which reduces the field's impact and its understanding of mechanisms. Here, we present an efficient way to interrogate heterogeneity and address these gaps in knowledge. This method a) presents scenarios that vividly represent different moderating contexts, b) measures a short-term behavioral outcome (e.g., an academic choice) that is known to relate to typical intervention outcomes (e.g., academic achievement), and c) assesses the causal effect of the moderating context on the link between the psychological variable typically targeted by interventions and this short-term outcome. We illustrated the utility of this approach across four experiments (total n = 3,235) that directly tested contextual moderators of the links between growth mindset, which is the belief that ability can be developed, and students' academic choices. The present results showed that teachers' growth mindset-supportive messages and the structural opportunities they provide moderated the link between students' mindsets and their choices (studies 1 to 3). This pattern was replicated in a nationally representative sample of adolescents and did not vary across demographic subgroups (study 2), nor was this pattern the result of several possible confounds (studies 3 to 4). Discussion centers on how this method of interrogating contextual heterogeneity can be applied to other behavioral science interventions and broaden their impact in other policy domains.
Academic Success , Students , Adolescent , Humans , Students/psychology , Schools , Educational Status
The two studies presented in this paper seek to resolve mixed findings in research linking activity of pubertal hormones to daily adolescent outcomes. In study 1 we used a series of Confirmatory Factor Analyses to compare the fit of one and two-factor models of seven steroid hormones (n = 994 participants, 8084 samples) of the HPA and HPG axes, using data from a field study (https://www.icpsr.umich.edu/web/ICPSR/studies/38180) collected over ten consecutive weekdays in a representative sample of teens starting high school. In study 2, we fit a Bayesian model to our large dataset to explore how hormone activity was related to outcomes that have been demonstrated to be linked to mental health and wellbeing (self-reports of daily affect and stress coping). Results reveal, first that a two-factor solution of adolescent hormones showed good fit to our data, and second, that HPG activity, rather than the more often examined HPA activity, was associated with improved daily affect ratios and stress coping. These findings suggest that field research, when it is combined with powerful statistical techniques, may help to improve our understanding of the relationship between adolescent hormones and daily measures of well-being.
Mental Health , Psychological Well-Being , Adolescent , Humans , Bayes Theorem , Hormones , Machine Learning , Stress, Psychological/psychology
There is growing policy interest in identifying contexts that cultivate self-regulation. Doing so often entails comparing groups of individuals (e.g., from different schools). We show that self-report questionnaires-the most prevalent modality for assessing self-regulation-are prone to reference bias, defined as systematic error arising from differences in the implicit standards by which individuals evaluate behavior. In three studies, adolescents (N = 229,685) whose peers performed better academically rated themselves lower in self-regulation and held higher standards for self-regulation. This effect was not observed for task measures of self-regulation and led to paradoxical predictions of college persistence 6 years later. These findings suggest that standards for self-regulation vary by social group, limiting the policy applications of self-report questionnaires.
Policy , Schools , Adolescent , Humans , Self Report , Bias , Surveys and Questionnaires
Mindset interventions, which shift students' beliefs about classroom experiences, have shown promise for promoting diversity in science, technology, engineering, and mathematics (STEM). Psychologists have emphasized the importance of customizing these interventions to specific courses, but there is not yet a protocol for doing so. We developed a protocol for creating customized "peer-modeled" mindset interventions that elicit advice from former students in videotaped interviews. In intervention activities, clips from these interviews, in which the former students' stories model the changes in thinking about challenge and struggle that helped them succeed in a specific course, are provided to incoming life sciences students. Using this protocol, we developed a customized intervention for three sections of Introductory Biology I at a large university and tested it in a randomized controlled trial (N = 917). The intervention shifted students' attributions for struggle in the class away from a lack of potential to succeed and toward the need to develop a better approach to studying. The intervention also improved students' approaches to studying and sense of belonging and had promising effects on performance and persistence in biology. Effects were pronounced among first-generation college students and underrepresented racial/ethnic minority students, who have been historically underrepresented in the STEM fields.
Biological Science Disciplines , Students , Humans , Ethnicity , Minority Groups , Engineering/education
Social-evaluative stressors-experiences in which people feel they could be judged negatively-pose a major threat to adolescent mental health1-3 and can cause young people to disengage from stressful pursuits, resulting in missed opportunities to acquire valuable skills. Here we show that replicable benefits for the stress responses of adolescents can be achieved with a short (around 30-min), scalable 'synergistic mindsets' intervention. This intervention, which is a self-administered online training module, synergistically targets both growth mindsets4 (the idea that intelligence can be developed) and stress-can-be-enhancing mindsets5 (the idea that one's physiological stress response can fuel optimal performance). In six double-blind, randomized, controlled experiments that were conducted with secondary and post-secondary students in the United States, the synergistic mindsets intervention improved stress-related cognitions (study 1, n = 2,717; study 2, n = 755), cardiovascular reactivity (study 3, n = 160; study 4, n = 200), daily cortisol levels (study 5, n = 118 students, n = 1,213 observations), psychological well-being (studies 4 and 5), academic success (study 5) and anxiety symptoms during the 2020 COVID-19 lockdowns (study 6, n = 341). Heterogeneity analyses (studies 3, 5 and 6) and a four-cell experiment (study 4) showed that the benefits of the intervention depended on addressing both mindsets-growth and stress-synergistically. Confidence in these conclusions comes from a conservative, Bayesian machine-learning statistical method for detecting heterogeneous effects6. Thus, our research has identified a treatment for adolescent stress that could, in principle, be scaled nationally at low cost.
Internet-Based Intervention , Psychology, Adolescent , Stress, Psychological , Academic Success , Adolescent , Anxiety/prevention & control , Bayes Theorem , COVID-19 , Cardiovascular Physiological Phenomena , Cognition , Double-Blind Method , Humans , Hydrocortisone/analysis , Machine Learning , Mental Health , Quarantine/psychology , Self Administration , Stress, Psychological/prevention & control , Stress, Psychological/psychology , Stress, Psychological/therapy , Students/psychology , United States
Background: Methodological comparisons of hormone quantification techniques have repeatedly demonstrated that, in adults, enzyme immunoassay (EIA) inflates steroid hormone concentrations relative to mass spectrometry. However, methodological comparisons in adolescent samples remain rare, and few studies have examined how chemiluminescent immunoassay (CLIA), another popular immunoassay, compares to mass spectrometry. Additionally, no studies have examined how differences in analytical techniques may be affecting relationships between steroid hormone levels and outcomes of interest, such as psychopathology. This pre-registered analysis of an existing dataset measured salivary cortisol and testosterone using both CLIA and liquid chromatography dual mass spectrometry (LC-MS/MS) in a repeated measures (516 samples) sample of 207 9th graders. Methods: In aim 1, this study sought to expand on past findings by 1) measuring inflation of testosterone and cortisol by CLIA in a relatively large adolescent sample, and 2) showing that CLIA (like EIA) testosterone inflation was especially true in groups with low 'true' testosterone levels. In aim 2, this study sought to examine the impact of hormone quantification method on relationships between hormone levels and psychopathological measures (the Children's Depression Inventory, the Perceived Social Stress Scale, the UCLA Loneliness Scale, and the Anxious Avoidant and Negative Self Evaluation subscales of the Social Anxiety Scale for Adolescents). Results: We found that CLIA, like EIA, inflated testosterone and cortisol levels and overestimated female testosterone resulting in suppressed sex differences in testosterone. We did not observe these same patterns when examining testosterone in individuals with differing levels of pubertal development. Results of psychopathology analyses demonstrated no significant method differences in hormone-psychopathology relationships. Conclusions: Our findings show that CLIA introduces proportional bias in cortisol and testosterone in a manner that suppresses sex differences in testosterone. Steroid measurement method did not significantly moderate the relationship between hormones and psychopathology in our sample, though more work is needed to investigate this question in larger, clinical samples.
Adolescents who hold an entity theory of personality - the belief that people cannot change - are more likely to report internalizing symptoms during the socially stressful transition to high school. It has been puzzling, however, why a cognitive belief about the potential for change predicts symptoms of an affective disorder. The present research integrated three models - implicit theories, hopelessness theories of depression, and the biopsychosocial model of challenge and threat - to shed light on this issue. Study 1 replicated the link between an entity theory and internalizing symptoms by synthesizing multiple datasets (N = 6,910). Study 2 examined potential mechanisms underlying this link using 8-month longitudinal data and 10-day diary reports during the stressful first year of high school (N = 533, 3,199 daily reports). The results showed that an entity theory of personality predicted increases in internalizing symptoms through tendencies to make fixed trait causal attributions about the self and maladaptive (i.e., "threat") stress appraisals. The findings support an integrative model whereby situation-general beliefs accumulate negative consequences for psychopathology via situation-specific attributions and appraisals.
Personality Disorders , Personality , Adolescent , Humans , Personality Disorders/psychology , Psychopathology , Schools , Social Perception
Adolescent females are at elevated risk for the development of depression. In this study, we addressed two questions: Are pubertal hormones associated with adolescent mental health? Might this association depend on pubertal development? We tested the hypothesis that estradiol, which has been associated with adolescent social sensitivity, might interact with pubertal stage to predict depression risk at three time points in ninth and tenth grade. Hormones and pubertal development were measured ninth-grade females. Linear regression analyses were used to predict fall ninth-grade (N = 79), spring ninth-grade (N = 76), and spring tenth-grade (N = 67) Children's Depression Inventory (CDI) scores. The hypothesized model was not statistically significant, but exploratory analyses revealed that two- and three-way interactions incorporating estradiol, puberty (stage and perceived onset), and cortisol predicted current and future CDI scores. Our exploratory model did not predict changes in CDI but did account for future (spring of ninth grade) CDI scores. Specifically, estradiol was positively correlated with fall and spring ninth-grade depressive symptoms in participants with high cortisol who also reported earlier stages and later perceived onset of pubertal development. These findings suggest that hormones associated with sensitivity to the social environment deserve consideration in models of adolescent depression risk.
Depression , Hydrocortisone , Adolescent , Child , Depression/diagnosis , Estradiol , Female , Humans , Puberty/psychology , Schools
A growth-mindset intervention teaches the belief that intellectual abilities can be developed. Where does the intervention work best? Prior research examined school-level moderators using data from the National Study of Learning Mindsets (NSLM), which delivered a short growth-mindset intervention during the first year of high school. In the present research, we used data from the NSLM to examine moderation by teachers' mindsets and answer a new question: Can students independently implement their growth mindsets in virtually any classroom culture, or must students' growth mindsets be supported by their teacher's own growth mindsets (i.e., the mindset-plus-supportive-context hypothesis)? The present analysis (9,167 student records matched with 223 math teachers) supported the latter hypothesis. This result stood up to potentially confounding teacher factors and to a conservative Bayesian analysis. Thus, sustaining growth-mindset effects may require contextual supports that allow the proffered beliefs to take root and flourish.
Schools , Students , Bayes Theorem , Humans , Learning , Mathematics
Beliefs play a central role in human development. For instance, a growth mindset-a belief about the malleability of intelligence-can shape how adolescents interpret and respond to academic difficulties and how they subsequently navigate the educational system. But do usually-adaptive beliefs have the same effects for adolescents regardless of the contexts they are in? Answering this question can reveal new insights into classic developmental questions about continuity and change. Here we present the Mindset×Context framework and we apply this model to the instructive case of growth mindset interventions. We show that teaching students a growth mindset is most effective in educational contexts that provide affordances for a growth mindset; that is, contexts that permit and encourage students to view ability as developable and to act on that belief. This evidence contradicts the "beliefs alone" hypothesis, which holds that teaching adolescents a growth mindset is enough and that students can profit from these beliefs in almost any context, even unsupportive ones. The Mindset×Context framework leads to the realization that in order to produce more widespread and lasting change, we must complement the belief-changing interventions that have been aimed at students with new interventions that guide teachers toward classroom policies and practices that allow students' growth mindset beliefs to take root and yield benefits.
Intelligence , Students , Adolescent , Humans , Problem Solving
In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.
Behavioral Sciences , Policy Making , Reproducibility of Results , Research Design , Artificial Intelligence , Behavior Control , Causality , Humans , Scientific Experimental Error
