A national experiment reveals where a growth mindset improves achievement.

A global priority for the behavioural sciences is to develop cost-effective, scalable interventions that could improve the academic outcomes of adolescents at a population level, but no such interventions have so far been evaluated in a population-generalizable sample. Here we show that a short (less than one hour), online growth mindset intervention-which teaches that intellectual abilities can be developed-improved grades among lower-achieving students and increased overall enrolment to advanced mathematics courses in a nationally representative sample of students in secondary education in the United States. Notably, the study identified school contexts that sustained the effects of the growth mindset intervention: the intervention changed grades when peer norms aligned with the messages of the intervention. Confidence in the conclusions of this study comes from independent data collection and processing, pre-registration of analyses, and corroboration of results by a blinded Bayesian analysis.

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis.

BACKGROUND: Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists. METHODS: This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland-Altman studies. RESULTS: After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (-41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (-45.0 % to 35.1 % and -54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (-14.2 % to 21.4 %). CONCLUSIONS: GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

Meta-analysis with Robust Variance Estimation: Expanding the Range of Working Models.

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the exact form of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multilevel and multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer potential benefits in terms of better capturing the types of data structures that occur in practice and, under some circumstances, improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the "metafor" and "clubSandwich" packages for R), illustrate the proposed approach in a meta-analysis of randomized trials on the effects of brief alcohol interventions for adolescents and young adults, and report findings from a simulation study evaluating the performance of the new methods.

Neurocognitive development and capabilities in boys with 49,XXXXY syndrome.

49,XXXXY was previously associated with profound to severe intellectual deficits. However, prior research papers on the cognitive profiles of this population were confounded by small samples sizes, wide age spreads, and incomplete histories of testosterone replacement therapy. This study is the first comprehensive, international investigation of the neurocognitive aspects of 49,XXXXY, and the potential effects of biological treatment on this profile. Sixty-seven boys from infancy to 11 years of age were enrolled in this longitudinal study, with the majority of boys postnatally diagnosed though chromosomal analysis. These boys received a comprehensive neurocognitive evaluation tailored to specific language-based deficits and cognitive challenges. Results revealed higher neurocognitive capacities, both verbally and nonverbally, than previously reported in this disorder. Infant boys with 49,XXXXY who received early hormonal therapy (EHT) had significantly higher scores on the cognitive domain of the Bayley Scales of Infant Development than untreated infants (p = .013). In addition, treated school-aged participants had significantly better scaled scores than untreated boys in form completion (p = .042), a task that requires deductive reasoning, on nonverbal testing on the Leiter International Performance Scales. This study indicates greater cognitive capacities with a wide range of abilities in the child with 49,XXXXY, thus warranting further investigation to identify and understand the critical influences on the etiology and the variability of those capacities.

Accuracy and precision of zero-heat-flux temperature measurements with the 3M™ Bair Hugger™ Temperature Monitoring System: a systematic review and meta-analysis.

Zero-heat-flux thermometers provide clinicians with the ability to continuously and non-invasively monitor body temperature. These devices are increasingly being used to substitute for more invasive core temperature measurements during surgery and in critical care. The aim of this review was to determine the accuracy and precision of zero-heat-flux temperature measurements from the 3M™ Bair Hugger™ Temperature Monitoring System. Medline and EMBASE were searched for studies that reported on a measurement of core or peripheral temperature that coincided with a measurement from the zero-heat-flux device. Study selection and quality assessment was performed independently using the Revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2). The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach was used to summarize the strength of the evidence. Pooled estimates of the mean bias and limits of agreement with outer 95% confidence intervals (population limits of agreement) were calculated. Sixteen studies were included. The primary meta-analysis of zero-heat-flux versus core temperature consisted of 22 comparisons from 16 individual studies. Data from 952 participants with 314,137 paired measurements were included. The pooled estimate for the mean bias was 0.03 °C. Population limits of agreement, which take into consideration the between-study heterogeneity and sampling error, were wide, spanning from - 0.93 to 0.98 °C. The GRADE evidence quality rating was downgraded to moderate due to concerns about study limitations. Population limits of agreement for the sensitivity analysis restricted to studies rated as having low risk of bias across all the domains of the QUADAS-2 were similar to the primary analysis. The range of uncertainty in the accuracy of a thermometer should be taken into account when using this device to inform clinical decision-making. Clinicians should therefore consider the potential that a temperature measurement from a 3M™ Bair Hugger™ Temperature Monitoring System could be as much as 1 °C higher or lower than core temperature. Use of this device may not be appropriate in situations where a difference in temperature of less than 1 °C is important to detect.

A review of the intriguing interaction between testosterone and neurocognitive development in males with 47,XXY.

PURPOSE OF REVIEW: Although 47,XXY (Klinefelter syndrome) was first discovered more than 50 years ago, there have been limited comprehensive studies on this disorder. The present review explains the study of neurodevelopmental dysfunction and the impact of testosterone replacement at specific junctions in the life of males with 47,XXY. The intricate relationship between testosterone, neurodevelopment, health, and well being warrants an in-depth investigation in order to achieve optimal outcomes. RECENT FINDINGS: Current literature suggests that the implementation of biological treatment has a positive impact on numerous areas of neurodevelopment. Further research is needed to determine ideal dosage, timing, and frequency of biological treatment for efficacy and safety of the child with 47,XXY. SUMMARY: As noninvasive prenatal screening has detected increasing numbers of fetuses with 47,XXY, parents may benefit from both prenatal and postnatal counseling, including the latest innovative biological treatment, that may further optimize the child's outcome, especially when coupled with targeted early intervention services.

Why Physicians Prescribe Prophylactic Seizure Medications after Intracerebral Hemorrhage: An Adaptive Conjoint Analysis.

BACKGROUND: Seizures are a morbid complication of intracerebral hemorrhage (ICH) and increase the risk for herniation, status epilepticus, and worse patient outcomes. Prophylactic levetiracetam is administered to approximately 40% of patients with ICH. It is unclear which patients are consciously selected for treatment by physicians. We sought to determine how patients are selected for treatment with prophylactic levetiracetam after ICH. METHODS: We administered an adaptive conjoint analysis using decision making software to an NIH Stroke Trials Network Working Group. The adaptive conjoint analysis determines the most influential attributes for making a decision in an iterative, algorithm-driven process. We asked respondents which would most influence a decision to administer prophylactic levetiracetam. The attributes and their levels were taken from published phenotypes associated with prophylactic seizure medications and the likelihood of seizures after ICH: hematoma location (lobar or basal ganglia), hematoma volume (<=10 mL or >10 mL), level of consciousness (Glasgow Coma Scale 5-12 or Glasgow Coma Scale 13-15), age (<65 or ≥65 years), and race (White or Caucasian or Black/African American). The algorithm terminated when the attributes were ranked from most to least influential. RESULTS: The study sample included 27 respondents who completed the adaptive conjoint analysis out of 42 who responded to the survey with a mean age of 43.4 ± 9.4 years. The attribute with the greatest weight was hematoma location (30%), followed by reduced level of consciousness (24%), hematoma volume (19%), race (14%), and age (13%). Ranks of attributes were different (P < .001). CONCLUSIONS: The decision to administer prophylactic levetiracetam to patients with ICH is driven by lobar hematoma location and depressed level of consciousness. Future research on prophylactic seizure medication could focus on patients most likely to receive it.

Accuracy and precision of transcutaneous carbon dioxide monitoring: a systematic review and meta-analysis.

BACKGROUND: Transcutaneous carbon dioxide (TcCO2) monitoring is a non-invasive alternative to arterial blood sampling. The aim of this review was to determine the accuracy and precision of TcCO2 measurements. METHODS: Medline and EMBASE (2000-2016) were searched for studies that reported on a measurement of PaCO2 that coincided with a measurement of TcCO2. Study selection and quality assessment (using the revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2)) were performed independently. The Grading Quality of Evidence and Strength of Recommendation approach was used to summarise the strength of the body of evidence. Pooled estimates of the mean bias between TcCO2 and PaCO2 and limits of agreement with outer 95% CIs (termed population limits of agreement) were calculated. RESULTS: The mean bias was -0.1 mm Hg and the population limits of agreement were -15 to 15 mm Hg for 7021 paired measurements taken from 2817 participants in 73 studies, which was outside of the clinically acceptable range (7.5 mm Hg). The lowest PaCO2 reported in the studies was 18 mm Hg and the highest was 103 mm Hg. The major sources of inconsistency were sensor location and temperature. The population limits of agreement were within the clinically acceptable range across 3974 paired measurements from 1786 participants in 44 studies that applied the sensor to the earlobe using the TOSCA and Sentec devices (-6 to 6 mm Hg). CONCLUSION: There are substantial differences between TcCO2 and PaCO2 depending on the context in which this technology is used. TcCO2 sensors should preferentially be applied to the earlobe and users should consider setting the temperature of the sensor higher than 42°C when monitoring at other sites. SYSTEMATIC REVIEW REGISTRATION NUMBER: PROSPERO; CRD42017057450.

A framework for the meta-analysis of Bland-Altman studies based on a limits of agreement approach.

Bland-Altman method comparison studies are common in the medical sciences and are used to compare a new measure to a gold-standard (often costlier or more invasive) measure. The distribution of these differences is summarized by two statistics, the 'bias' and standard deviation, and these measures are combined to provide estimates of the limits of agreement (LoA). When these LoA are within the bounds of clinically insignificant differences, the new non-invasive measure is preferred. Very often, multiple Bland-Altman studies have been conducted comparing the same two measures, and random-effects meta-analysis provides a means to pool these estimates. We provide a framework for the meta-analysis of Bland-Altman studies, including methods for estimating the LoA and measures of uncertainty (i.e., confidence intervals). Importantly, these LoA are likely to be wider than those typically reported in Bland-Altman meta-analyses. Frequently, Bland-Altman studies report results based on repeated measures designs but do not properly adjust for this design in the analysis. Meta-analyses of Bland-Altman studies frequently exclude these studies for this reason. We provide a meta-analytic approach that allows inclusion of estimates from these studies. This includes adjustments to the estimate of the standard deviation and a method for pooling the estimates based upon robust variance estimation. An example is included based on a previously published meta-analysis. Copyright © 2017 John Wiley & Sons, Ltd.

Testing an Integrated Model of Program Implementation: the Food, Health & Choices School-Based Childhood Obesity Prevention Intervention Process Evaluation.

Childhood obesity is a complex, worldwide problem. Significant resources are invested in its prevention, and high-quality evaluations of these efforts are important. Conducting trials in school settings is complicated, making process evaluations useful for explaining results. Intervention fidelity has been demonstrated to influence outcomes, but others have suggested that other aspects of implementation, including participant responsiveness, should be examined more systematically. During Food, Health & Choices (FHC), a school-based childhood obesity prevention trial designed to test a curriculum and wellness policy taught by trained FHC instructors to fifth grade students in 20 schools during 2012-2013, we assessed relationships among facilitator behaviors (i.e., fidelity and teacher interest); participant behaviors (i.e., student satisfaction and recall); and program outcomes (i.e., energy balance-related behaviors) using hierarchical linear models, controlling for student, class, and school characteristics. We found positive relationships between student satisfaction and recall and program outcomes, but not fidelity and program outcomes. We also found relationships between teacher interest and fidelity when teachers participated in implementation. Finally, we found a significant interaction between fidelity and satisfaction on behavioral outcomes. These findings suggest that individual students in the same class responded differently to the same intervention. They also suggest the importance of teacher buy-in for successful intervention implementation. Future studies should examine how facilitator and participant behaviors together are related to both outcomes and implementation. Assessing multiple aspects of implementation using models that account for contextual influences on behavioral outcomes is an important step forward for prevention intervention process evaluations.

The Recline Exercise: Comparisons with the Head Lift Exercise in Healthy Adults.

The aim of this investigation was to examine the comparative effectiveness of the new Recline Exercise (RE) and the traditional Head Lift Exercise (Shaker Exercise) on submental muscle activity, tongue strength, and perceived exertion in 40 healthy young adults (mean age = 24.5 years, SD 2.6 years). Both groups participated in a 6-week exercise regimen. Outcome variables evaluated pre- and post-exercise included: duration and peak amplitude of submental muscle activity during swallowing measured via surface electromyography (sEMG); anterior and posterior isometric lingual pressures measured with the Iowa Oral Performance Instrument; and perceived exertion levels measured with the Borg category-ratio scale of perceived exertion. Results indicated no significant pre-post differences within or between groups in swallow duration and peak amplitude. In addition, the RE group demonstrated significant post-treatment increases in anterior and posterior tongue strength [p = 0.009; p < 0.001]; however, these increases were of small magnitude (d = 0.132; d = 0.319). Both groups showed marked improvements in perceived exertion levels [p < 0.001]. Our findings indicate that healthy young adults who perform the RE or the HLE do not have significant swallow duration or amplitude gains, most likely due to the reduced need for such gains in the healthy head/neck musculature for submaximal tasks. Furthermore, the significant lingual strength gains seen with the RE indicate that additional musculature is being engaged during its completion. These results are encouraging; however, future research in older adults and patients with dysphagia with examination of swallowing biomechanics is needed to determine its full potential as a rehabilitative regimen.

Enhancing recall in automated record screening: A resampling algorithm.

Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with high priority. In previous studies, screening prioritization is often referred to as automatic literature screening or automatic literature identification. Numerous screening prioritization methods have been proposed in recent years. However, there is a lack of screening prioritization methods with reliable performance. Our objective is to develop a screening prioritization algorithm with reliable performance for practical use, for example, an algorithm that guarantees an 80% chance of identifying at least 80 % of the relevant records. Based on a target-based method proposed in Cormack and Grossman, we propose a screening prioritization algorithm using sampling with replacement. The algorithm is a wrapper algorithm that can work with any current screening prioritization algorithm to guarantee the performance. We prove, with mathematics and probability theory, that the algorithm guarantees the performance. We also run numeric experiments to test the performance of our algorithm when applied in practice. The numeric experiment results show this algorithm achieve reliable performance under different circumstances. The proposed screening prioritization algorithm can be reliably used in real world research synthesis tasks.

Effect sizes in ANCOVA and difference-in-differences designs.

It is common practice in both randomized and quasi-experiments to adjust for baseline characteristics when estimating the average effect of an intervention. The inclusion of a pre-test, for example, can reduce both the standard error of this estimate and-in non-randomized designs-its bias. At the same time, it is also standard to report the effect of an intervention in standardized effect size units, thereby making it comparable to other interventions and studies. Curiously, the estimation of this effect size, including covariate adjustment, has received little attention. In this article, we provide a framework for defining effect sizes in designs with a pre-test (e.g., difference-in-differences and analysis of covariance) and propose estimators of those effect sizes. The estimators and approximations to their sampling distributions are evaluated using a simulation study and then demonstrated using an example from published data.

Why Meta-Analyses of Growth Mindset and Other Interventions Should Follow Best Practices for Examining Heterogeneity: Commentary on Macnamara and Burgoyne (2023) and Burnette et al. (2023).

Meta-analysts often ask a yes-or-no question: Is there an intervention effect or not? This traditional, all-or-nothing thinking stands in contrast with current best practice in meta-analysis, which calls for a heterogeneity-attuned approach (i.e., focused on the extent to which effects vary across procedures, participant groups, or contexts). This heterogeneity-attuned approach allows researchers to understand where effects are weaker or stronger and reveals mechanisms. The current article builds on a rare opportunity to compare two recent meta-analyses that examined the same literature (growth mindset interventions) but used different methods and reached different conclusions. One meta-analysis used a traditional approach (Macnamara and Burgoyne, in press), which aggregated effect sizes for each study before combining them and examined moderators one-by-one by splitting the data into small subgroups. The second meta-analysis (Burnette et al., in press) modeled the variation of effects within studies-across subgroups and outcomes-and applied modern, multi-level meta-regression methods. The former concluded that growth mindset effects are biased, but the latter yielded nuanced conclusions consistent with theoretical predictions. We explain why the practices followed by the latter meta-analysis were more in line with best practices for analyzing large and heterogeneous literatures. Further, an exploratory re-analysis of the data showed that applying the modern, heterogeneity-attuned methods from Burnette et al. (in press) to the dataset employed by Macnamara and Burgoyne (in press) confirmed Burnette et al.'s conclusions; namely, that there was a meaningful, significant effect of growth mindset in focal (at-risk) groups. This article concludes that heterogeneity-attuned meta-analysis is important both for advancing theory and for avoiding the boom-or-bust cycle that plagues too much of psychological science.

A machine learning analysis of the relationship of demographics and social gathering attendance from 41 countries during pandemic.

Knowing who to target with certain messages is the prerequisite of efficient public health campaigns during pandemics. Using the COVID-19 pandemic situation, we explored which facets of the society-defined by age, gender, income, and education levels-are the most likely to visit social gatherings and aggravate the spread of a disease. Analyzing the reported behavior of 87,169 individuals from 41 countries, we found that in the majority of the countries, the proportion of social gathering-goers was higher in male than female, younger than older, lower-educated than higher educated, and low-income than high-income subgroups of the populations. However, the data showed noteworthy heterogeneity between the countries warranting against generalizing from one country to another. The analysis also revealed that relative to other demographic factors, income was the strongest predictor of avoidance of social gatherings followed by age, education, and gender. Although the observed strength of these associations was relatively small, we argue that incorporating demographic-based segmentation into public health campaigns can increase the efficiency of campaigns with an important caveat: the exploration of these associations needs to be done on a country level before using the information to target populations in behavior change interventions.

Accuracy and Precision of Energy Expenditure, Heart Rate, and Steps Measured by Combined-Sensing Fitbits Against Reference Measures: Systematic Review and Meta-analysis.

BACKGROUND: Although it is widely recognized that physical activity is an important determinant of health, assessing this complex behavior is a considerable challenge. OBJECTIVE: The purpose of this systematic review and meta-analysis is to examine, quantify, and report the current state of evidence for the validity of energy expenditure, heart rate, and steps measured by recent combined-sensing Fitbits. METHODS: We conducted a systematic review and Bland-Altman meta-analysis of validation studies of combined-sensing Fitbits against reference measures of energy expenditure, heart rate, and steps. RESULTS: A total of 52 studies were included in the systematic review. Among the 52 studies, 41 (79%) were included in the meta-analysis, representing 203 individual comparisons between Fitbit devices and a criterion measure (ie, n=117, 57.6% for heart rate; n=49, 24.1% for energy expenditure; and n=37, 18.2% for steps). Overall, most authors of the included studies concluded that recent Fitbit models underestimate heart rate, energy expenditure, and steps compared with criterion measures. These independent conclusions aligned with the results of the pooled meta-analyses showing an average underestimation of -2.99 beats per minute (k comparison=74), -2.77 kcal per minute (k comparison=29), and -3.11 steps per minute (k comparison=19), respectively, of the Fitbit compared with the criterion measure (results obtained after removing the high risk of bias studies; population limit of agreements for heart rate, energy expenditure, and steps: -23.99 to 18.01, -12.75 to 7.41, and -13.07 to 6.86, respectively). CONCLUSIONS: Fitbit devices are likely to underestimate heart rate, energy expenditure, and steps. The estimation of these measurements varied by the quality of the study, age of the participants, type of activities, and the model of Fitbit. The qualitative conclusions of most studies aligned with the results of the meta-analysis. Although the expected level of accuracy might vary from one context to another, this underestimation can be acceptable, on average, for steps and heart rate. However, the measurement of energy expenditure may be inaccurate for some research purposes.

Factors Determining the Agreement between Aerobic Threshold and Point of Maximal Fat Oxidation: Follow-Up on a Systematic Review and Meta-Analysis on Association.

Regular exercise at the intensity matching maximal fat oxidation (FATmax) has been proposed as a key element in both athletes and clinical populations when aiming to enhance the body's ability to oxidize fat. In order to allow a more standardized and tailored training approach, the connection between FATmax and the individual aerobic thresholds (AerT) has been examined. Although recent findings strongly suggest that a relationship exists between these two intensities, correlation alone is not sufficient to confirm that the intensities necessarily coincide and that the error between the two measures is small. Thus, this systematic review and meta-analysis aim to examine the agreement levels between the exercise intensities matching FATmax and AerT by pooling limits of agreement in a function of three parameters: (i) the average difference, (ii) the average within-study variation, and (iii) the variation in bias across studies, and to examine the influence of clinical and methodological inter- and intra-study differences on agreement levels. This study was registered with PROSPERO (CRD42021239351) and ClinicalTrials (NCT03789045). PubMed and Google Scholar were searched for studies examining FATmax and AerT connection. Overall, 12 studies with forty-five effect sizes and a total of 774 subjects fulfilled the inclusion criteria. The ROBIS tool for risk of bias assessment was used to determine the quality of included studies. In conclusion, the overall 95% limits of agreement of the differences between FATmax and AerT exercise intensities were larger than the a priori determined acceptable agreement due to the large variance caused by clinical and methodological differences among the studies. Therefore, we recommend that future studies follow a strict standardization of data collection and analysis of FATmax- and AerT-related outcomes.

Case Report: A Case Study on the Neurodevelopmental Profile of a Child With Pallister-Killian Syndrome and His Unaffected Twin.

Pallister-Killian syndrome is an uncommon genetic disorder that has broad developmental and multisystemic effects. While medical complications are widely reported throughout the literature, research on the neurodevelopmental profile has been limited. Case reports make up the majority of the few existing studies regarding the neurodevelopmental phenotype associated with this disorder. The current case report describes a 3-year-old male with Pallister-Killian syndrome (AF), reports the neurodevelopmental evaluation of his unaffected twin brother (MF), and outlines the results of an optical imaging study on both boys. AF presents with severe developmental delays, however, he ambulates with support and engages in conversation using his communication device. Most severely impaired was AF's speech and expressive language, with childhood apraxia of speech (CAS) as a possible explanation for these severe deficits. MF, the sibling, demonstrated neurotypical abilities and often advanced scores for his age. Both subjects completed a functional near-infrared spectroscopy (fNIRS) study, revealing decreased temporal and frontal lobe function in AF and typical functioning in MF. This case report expands on the existing literature on PKS by describing variances in fraternal twin presentation and novel reporting on fNIRS findings in both boys.

Behavioural science is unlikely to change the world without a heterogeneity revolution.

In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.