Megastudies improve the impact of applied behavioural science.

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes1. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals2. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research3-6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies.

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Algorithms as discrimination detectors.

Preventing discrimination requires that we have means of detecting it, and this can be enormously difficult when human beings are making the underlying decisions. As applied today, algorithms can increase the risk of discrimination. But as we argue here, algorithms by their nature require a far greater level of specificity than is usually possible with human decision making, and this specificity makes it possible to probe aspects of the decision in additional ways. With the right changes to legal and regulatory systems, algorithms can thus potentially make it easier to detect-and hence to help prevent-discrimination.

EVALUATING CONTRADICTORY EXPERIMENTAL AND NON-EXPERIMENTAL ESTIMATES OF NEIGHBORHOOD EFFECTS ON ECONOMIC OUTCOMES FOR ADULTS.

Although non-experimental studies find robust neighborhood effects on adults, such findings have been challenged by results from the Moving to Opportunity (MTO) residential mobility experiment. Using a within-study comparison design, this paper compares experimental and non-experimental estimates from MTO and a parallel analysis of the Panel Study of Income Dynamics (PSID). Striking similarities were found between non-experimental estimates based on MTO and PSID. No clear evidence was found that different estimates are related to duration of adult exposure to disadvantaged neighborhoods, non-linear effects of neighborhood conditions, magnitude of the change in neighborhood context, frequency of moves, treatment effect heterogeneity, or measurement, although uncertainty bands around our estimates were sometimes large. One other possibility is that MTO-induced moves might have been unusually disruptive, but results are inconsistent for that hypothesis. Taken together, the findings suggest that selection bias might account for evidence of neighborhood effects on adult economic outcomes in non-experimental studies.

Neighborhood effects on use of African-American Vernacular English.

African-American Vernacular English (AAVE) is systematic, rooted in history, and important as an identity marker and expressive resource for its speakers. In these respects, it resembles other vernacular or nonstandard varieties, like Cockney or Appalachian English. But like them, AAVE can trigger discrimination in the workplace, housing market, and schools. Understanding what shapes the relative use of AAVE vs. Standard American English (SAE) is important for policy and scientific reasons. This work presents, to our knowledge, the first experimental estimates of the effects of moving into lower-poverty neighborhoods on AAVE use. We use data on non-Hispanic African-American youth (n = 629) from a large-scale, randomized residential mobility experiment called Moving to Opportunity (MTO), which enrolled a sample of mostly minority families originally living in distressed public housing. Audio recordings of the youth were transcribed and coded for the use of five grammatical and five phonological AAVE features to construct a measure of the proportion of possible instances, or tokens, in which speakers use AAVE rather than SAE speech features. Random assignment to receive a housing voucher to move into a lower-poverty area (the intention-to-treat effect) led youth to live in neighborhoods (census tracts) with an 11 percentage point lower poverty rate on average over the next 10-15 y and reduced the share of AAVE tokens by ∼3 percentage points compared with the MTO control group youth. The MTO effect on AAVE use equals approximately half of the difference in AAVE frequency observed between youth whose parents have a high school diploma and those whose parents do not.

Neighborhoods, obesity, and diabetes--a randomized social experiment.

BACKGROUND: The question of whether neighborhood environment contributes directly to the development of obesity and diabetes remains unresolved. The study reported on here uses data from a social experiment to assess the association of randomly assigned variation in neighborhood conditions with obesity and diabetes. METHODS: From 1994 through 1998, the Department of Housing and Urban Development (HUD) randomly assigned 4498 women with children living in public housing in high-poverty urban census tracts (in which ≥40% of residents had incomes below the federal poverty threshold) to one of three groups: 1788 were assigned to receive housing vouchers, which were redeemable only if they moved to a low-poverty census tract (where <10% of residents were poor), and counseling on moving; 1312 were assigned to receive unrestricted, traditional vouchers, with no special counseling on moving; and 1398 were assigned to a control group that was offered neither of these opportunities. From 2008 through 2010, as part of a long-term follow-up survey, we measured data indicating health outcomes, including height, weight, and level of glycated hemoglobin (HbA(1c)). RESULTS: As part of our long-term survey, we obtained data on body-mass index (BMI, the weight in kilograms divided by the square of the height in meters) for 84.2% of participants and data on glycated hemoglobin level for 71.3% of participants. Response rates were similar across randomized groups. The prevalences of a BMI of 35 or more, a BMI of 40 or more, and a glycated hemoglobin level of 6.5% or more were lower in the group receiving the low-poverty vouchers than in the control group, with an absolute difference of 4.61 percentage points (95% confidence interval [CI], -8.54 to -0.69), 3.38 percentage points (95% CI, -6.39 to -0.36), and 4.31 percentage points (95% CI, -7.82 to -0.80), respectively. The differences between the group receiving traditional vouchers and the control group were not significant. CONCLUSIONS: The opportunity to move from a neighborhood with a high level of poverty to one with a lower level of poverty was associated with modest but potentially important reductions in the prevalence of extreme obesity and diabetes. The mechanisms underlying these associations remain unclear but warrant further investigation, given their potential to guide the design of community-level interventions intended to improve health. (Funded by HUD and others.).

π-conjugated polymer-fullerene covalent hybrids via ambient conditions Diels-Alder ligation.

The established ability of graphitic carbon-nanomaterials to undergo ambient condition Diels-Alder reactions with cyclopentadienyl (Cp) groups is herein employed to prepare fullerene-polythiophene covalent hybrids with improved electron transfer and film forming characteristics. A novel precisely designed polythiophene (M n 9.8 kD, D 1.4) with 17 mol% of Cp-groups bearing repeat unit is prepared via Grignard metathesis polymerization. The UV/Vis absorption and fluorescence (λex 450 nm) characteristics of polythiophene with pendant Cp-groups (λmax 447 nm, λe-max 576 nm) are comparable to the reference poly(3-hexylthiophene) (λmax 450 nm, λe-max 576 nm). The novel polythiophene with pendant Cp-groups is capable of producing solvent-stable free-standing polythiophene films, and non-solvent assisted self-assemblies resulting in solvent-stable nanoporous-microstructures. (1) H-NMR spectroscopy reveals an efficient reaction of the pendant Cp-groups with C60 . The UV/Vis spectroscopic analyses of solution and thin films of the covalent and physical hybrids disclose closer donor-acceptor packing in the case of covalent hybrids. AFM images evidence that the covalent hybrids form smooth films with finer lamellar-organization. The effect is particularly remarkable in the case of poorly soluble C60 . A significant enhancement in photo-voltage is observed for all devices constituted of covalent hybrids, highlighting novel avenues to developing efficient electron donor-acceptor combinations for light harvesting systems.

Associations of housing mobility interventions for children in high-poverty neighborhoods with subsequent mental disorders during adolescence.

IMPORTANCE: Youth in high-poverty neighborhoods have high rates of emotional problems. Understanding neighborhood influences on mental health is crucial for designing neighborhood-level interventions. OBJECTIVE: To perform an exploratory analysis of associations between housing mobility interventions for children in high-poverty neighborhoods and subsequent mental disorders during adolescence. DESIGN, SETTING, AND PARTICIPANTS: The Moving to Opportunity Demonstration from 1994 to 1998 randomized 4604 volunteer public housing families with 3689 children in high-poverty neighborhoods into 1 of 2 housing mobility intervention groups (a low-poverty voucher group vs a traditional voucher group) or a control group. The low-poverty voucher group (n=1430) received vouchers to move to low-poverty neighborhoods with enhanced mobility counseling. The traditional voucher group (n=1081) received geographically unrestricted vouchers. Controls (n=1178) received no intervention. Follow-up evaluation was performed 10 to 15 years later (June 2008-April 2010) with participants aged 13 to 19 years (0-8 years at randomization). Response rates were 86.9% to 92.9%. MAIN OUTCOMES AND MEASURES: Presence of mental disorders from the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) within the past 12 months, including major depressive disorder, panic disorder, posttraumatic stress disorder (PTSD), oppositional-defiant disorder, intermittent explosive disorder, and conduct disorder, as assessed post hoc with a validated diagnostic interview. RESULTS: Of the 3689 adolescents randomized, 2872 were interviewed (1407 boys and 1465 girls). Compared with the control group, boys in the low-poverty voucher group had significantly increased rates of major depression (7.1% vs 3.5%; odds ratio (OR), 2.2 [95% CI, 1.2-3.9]), PTSD (6.2% vs 1.9%; OR, 3.4 [95% CI, 1.6-7.4]), and conduct disorder (6.4% vs 2.1%; OR, 3.1 [95% CI, 1.7-5.8]). Boys in the traditional voucher group had increased rates of PTSD compared with the control group (4.9% vs 1.9%, OR, 2.7 [95% CI, 1.2-5.8]). However, compared with the control group, girls in the traditional voucher group had decreased rates of major depression (6.5% vs 10.9%; OR, 0.6 [95% CI, 0.3-0.9]) and conduct disorder (0.3% vs 2.9%; OR, 0.1 [95% CI, 0.0-0.4]). CONCLUSIONS AND RELEVANCE: Interventions to encourage moving out of high-poverty neighborhoods were associated with increased rates of depression, PTSD, and conduct disorder among boys and reduced rates of depression and conduct disorder among girls. Better understanding of interactions among individual, family, and neighborhood risk factors is needed to guide future public housing policy changes.

Long-term effects of the Moving to Opportunity residential mobility experiment on crime and delinquency.

OBJECTIVES: Using data from a randomized experiment, to examine whether moving youth out of areas of concentrated poverty, where a disproportionate amount of crime occurs, prevents involvement in crime. METHODS: We draw on new administrative data from the U.S. Department of Housing and Urban Development's Moving to Opportunity (MTO) experiment. MTO families were randomized into an experimental group offered a housing voucher that could only be used to move to a low-poverty neighborhood, a Section 8 housing group offered a standard housing voucher, and a control group. This paper focuses on MTO youth ages 15-25 in 2001 (n = 4,643) and analyzes intention to treat effects on neighborhood characteristics and criminal behavior (number of violent- and property-crime arrests) through 10 years after randomization. RESULTS: We find the offer of a housing voucher generates large improvements in neighborhood conditions that attenuate over time and initially generates substantial reductions in violent-crime arrests and sizable increases in property-crime arrests for experimental group males. The crime effects attenuate over time along with differences in neighborhood conditions. CONCLUSIONS: Our findings suggest that criminal behavior is more strongly related to current neighborhood conditions (situational neighborhood effects) than to past neighborhood conditions (developmental neighborhood effects). The MTO design makes it difficult to determine which specific neighborhood characteristics are most important for criminal behavior. Our administrative data analyses could be affected by differences across areas in the likelihood that a crime results in an arrest.

The Inversion Problem: Why Algorithms Should Infer Mental State and Not Just Predict Behavior.

More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden-but serious-problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them "inversion problems": The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.

Anti-depressants and suicide.

Suicide takes the lives of around a million people each year, most of whom suffer from depression. In recent years there has been growing controversy about whether one of the best-selling anti-depressants - selective serotonin reuptake inhibitors (SSRIs) - increases or decreases the risk of completed suicide. Randomized clinical trials are not informative in this application because of small samples and other problems. We present what we believe are the most scientifically credible estimates to date on how SSRI sales affect suicide mortality using data from 26 countries for up to 25 years. We exploit just the variation in SSRI sales that can be explained by institutional differences in how drugs are regulated, priced, and distributed, as reflected by the sales growth of new drugs more generally. We find an increase in SSRI sales of 1 pill per capita (12% of 2000 sales levels) reduces suicide by 5%.

Long-term effects of head start on low-income children.

A growing body of research suggests that the first few years of life are a particularly promising time to intervene in the lives of low-income children, although the long-term effects on children of the U.S. government's primary early childhood program--Head Start--remains the topic of debate. In this article we review what is known about Head Start and argue that the program is likely to generate benefits to participants and society as a whole that are large enough to justify the program's costs. Although in principle there could be more beneficial ways of deploying Head Start resources, the benefits of such changes remain uncertain and there is some downside risk.

HUMAN DECISIONS AND MACHINE PREDICTIONS.

Can machine learning improve human decision making? Bail decisions provide a good test case. Millions of times each year, judges make jail-or-release decisions that hinge on a prediction of what a defendant would do if released. The concreteness of the prediction task combined with the volume of data available makes this a promising machine-learning application. Yet comparing the algorithm to judges proves complicated. First, the available data are generated by prior judge decisions. We only observe crime outcomes for released defendants, not for those judges detained. This makes it hard to evaluate counterfactual decision rules based on algorithmic predictions. Second, judges may have a broader set of preferences than the variable the algorithm predicts; for instance, judges may care specifically about violent crimes or about racial inequities. We deal with these problems using different econometric strategies, such as quasi-random assignment of cases to judges. Even accounting for these concerns, our results suggest potentially large welfare gains: one policy simulation shows crime reductions up to 24.7% with no change in jailing rates, or jailing rate reductions up to 41.9% with no increase in crime rates. Moreover, all categories of crime, including violent crimes, show reductions; and these gains can be achieved while simultaneously reducing racial disparities. These results suggest that while machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals. JEL Codes: C10 (Econometric and statistical methods and methodology), C55 (Large datasets: Modeling and analysis), K40 (Legal procedure, the legal system, and illegal behavior).

The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables.

Evaluating whether machines improve on human performance is one of the central questions of machine learning. However, there are many domains where the data is selectively labeled in the sense that the observed outcomes are themselves a consequence of the existing choices of the human decision-makers. For instance, in the context of judicial bail decisions, we observe the outcome of whether a defendant fails to return for their court appearance only if the human judge decides to release the defendant on bail. This selective labeling makes it harder to evaluate predictive models as the instances for which outcomes are observed do not represent a random sample of the population. Here we propose a novel framework for evaluating the performance of predictive models on selectively labeled data. We develop an approach called contraction which allows us to compare the performance of predictive models and human decision-makers without resorting to counterfactual inference. Our methodology harnesses the heterogeneity of human decision-makers and facilitates effective evaluation of predictive models even in the presence of unmeasured confounders (unobservables) which influence both human decisions and the resulting outcomes. Experimental results on real world datasets spanning diverse domains such as health care, insurance, and criminal justice demonstrate the utility of our evaluation metric in comparing human decisions and machine predictions.

Thinking, Fast and Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago.

We present the results of three large-scale randomized controlled trials (RCTs) carried out in Chicago, testing interventions to reduce crime and dropout by changing the decision making of economically disadvantaged youth. We study a program called Becoming a Man (BAM), developed by the nonprofit Youth Guidance, in two RCTs implemented in 2009-2010 and 2013-2015. In the two studies participation in the program reduced total arrests during the intervention period by 28-35%, reduced violent-crime arrests by 45-50%, improved school engagement, and in the first study where we have follow-up data, increased graduation rates by 12-19%. The third RCT tested a program with partially overlapping components carried out in the Cook County Juvenile Temporary Detention Center (JTDC), which reduced readmission rates to the facility by 21%. These large behavioral responses combined with modest program costs imply benefit-cost ratios for these interventions from 5-to-1 up to 30-to-1 or more. Our data on mechanisms are not ideal, but we find no positive evidence that these effects are due to changes in emotional intelligence or social skills, self-control or "grit," or a generic mentoring effect. We find suggestive support for the hypothesis that the programs work by helping youth slow down and reflect on whether their automatic thoughts and behaviors are well suited to the situation they are in, or whether the situation could be construed differently. JEL Codes: C91, C93, D03, D1, I24, I3, I32, K42.