The Inversion Problem: Why Algorithms Should Infer Mental State and Not Just Predict Behavior.

More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden-but serious-problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them "inversion problems": The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.

EVALUATING CONTRADICTORY EXPERIMENTAL AND NON-EXPERIMENTAL ESTIMATES OF NEIGHBORHOOD EFFECTS ON ECONOMIC OUTCOMES FOR ADULTS.

Although non-experimental studies find robust neighborhood effects on adults, such findings have been challenged by results from the Moving to Opportunity (MTO) residential mobility experiment. Using a within-study comparison design, this paper compares experimental and non-experimental estimates from MTO and a parallel analysis of the Panel Study of Income Dynamics (PSID). Striking similarities were found between non-experimental estimates based on MTO and PSID. No clear evidence was found that different estimates are related to duration of adult exposure to disadvantaged neighborhoods, non-linear effects of neighborhood conditions, magnitude of the change in neighborhood context, frequency of moves, treatment effect heterogeneity, or measurement, although uncertainty bands around our estimates were sometimes large. One other possibility is that MTO-induced moves might have been unusually disruptive, but results are inconsistent for that hypothesis. Taken together, the findings suggest that selection bias might account for evidence of neighborhood effects on adult economic outcomes in non-experimental studies.

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies.

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Megastudies improve the impact of applied behavioural science.

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes1. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals2. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research3-6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

Algorithms as discrimination detectors.

Preventing discrimination requires that we have means of detecting it, and this can be enormously difficult when human beings are making the underlying decisions. As applied today, algorithms can increase the risk of discrimination. But as we argue here, algorithms by their nature require a far greater level of specificity than is usually possible with human decision making, and this specificity makes it possible to probe aspects of the decision in additional ways. With the right changes to legal and regulatory systems, algorithms can thus potentially make it easier to detect-and hence to help prevent-discrimination.

HUMAN DECISIONS AND MACHINE PREDICTIONS.

Can machine learning improve human decision making? Bail decisions provide a good test case. Millions of times each year, judges make jail-or-release decisions that hinge on a prediction of what a defendant would do if released. The concreteness of the prediction task combined with the volume of data available makes this a promising machine-learning application. Yet comparing the algorithm to judges proves complicated. First, the available data are generated by prior judge decisions. We only observe crime outcomes for released defendants, not for those judges detained. This makes it hard to evaluate counterfactual decision rules based on algorithmic predictions. Second, judges may have a broader set of preferences than the variable the algorithm predicts; for instance, judges may care specifically about violent crimes or about racial inequities. We deal with these problems using different econometric strategies, such as quasi-random assignment of cases to judges. Even accounting for these concerns, our results suggest potentially large welfare gains: one policy simulation shows crime reductions up to 24.7% with no change in jailing rates, or jailing rate reductions up to 41.9% with no increase in crime rates. Moreover, all categories of crime, including violent crimes, show reductions; and these gains can be achieved while simultaneously reducing racial disparities. These results suggest that while machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals. JEL Codes: C10 (Econometric and statistical methods and methodology), C55 (Large datasets: Modeling and analysis), K40 (Legal procedure, the legal system, and illegal behavior).

Thinking, Fast and Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago.

We present the results of three large-scale randomized controlled trials (RCTs) carried out in Chicago, testing interventions to reduce crime and dropout by changing the decision making of economically disadvantaged youth. We study a program called Becoming a Man (BAM), developed by the nonprofit Youth Guidance, in two RCTs implemented in 2009-2010 and 2013-2015. In the two studies participation in the program reduced total arrests during the intervention period by 28-35%, reduced violent-crime arrests by 45-50%, improved school engagement, and in the first study where we have follow-up data, increased graduation rates by 12-19%. The third RCT tested a program with partially overlapping components carried out in the Cook County Juvenile Temporary Detention Center (JTDC), which reduced readmission rates to the facility by 21%. These large behavioral responses combined with modest program costs imply benefit-cost ratios for these interventions from 5-to-1 up to 30-to-1 or more. Our data on mechanisms are not ideal, but we find no positive evidence that these effects are due to changes in emotional intelligence or social skills, self-control or "grit," or a generic mentoring effect. We find suggestive support for the hypothesis that the programs work by helping youth slow down and reflect on whether their automatic thoughts and behaviors are well suited to the situation they are in, or whether the situation could be construed differently. JEL Codes: C91, C93, D03, D1, I24, I3, I32, K42.

The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables.

Evaluating whether machines improve on human performance is one of the central questions of machine learning. However, there are many domains where the data is selectively labeled in the sense that the observed outcomes are themselves a consequence of the existing choices of the human decision-makers. For instance, in the context of judicial bail decisions, we observe the outcome of whether a defendant fails to return for their court appearance only if the human judge decides to release the defendant on bail. This selective labeling makes it harder to evaluate predictive models as the instances for which outcomes are observed do not represent a random sample of the population. Here we propose a novel framework for evaluating the performance of predictive models on selectively labeled data. We develop an approach called contraction which allows us to compare the performance of predictive models and human decision-makers without resorting to counterfactual inference. Our methodology harnesses the heterogeneity of human decision-makers and facilitates effective evaluation of predictive models even in the presence of unmeasured confounders (unobservables) which influence both human decisions and the resulting outcomes. Experimental results on real world datasets spanning diverse domains such as health care, insurance, and criminal justice demonstrate the utility of our evaluation metric in comparing human decisions and machine predictions.

Neighborhood effects on use of African-American Vernacular English.

African-American Vernacular English (AAVE) is systematic, rooted in history, and important as an identity marker and expressive resource for its speakers. In these respects, it resembles other vernacular or nonstandard varieties, like Cockney or Appalachian English. But like them, AAVE can trigger discrimination in the workplace, housing market, and schools. Understanding what shapes the relative use of AAVE vs. Standard American English (SAE) is important for policy and scientific reasons. This work presents, to our knowledge, the first experimental estimates of the effects of moving into lower-poverty neighborhoods on AAVE use. We use data on non-Hispanic African-American youth (n = 629) from a large-scale, randomized residential mobility experiment called Moving to Opportunity (MTO), which enrolled a sample of mostly minority families originally living in distressed public housing. Audio recordings of the youth were transcribed and coded for the use of five grammatical and five phonological AAVE features to construct a measure of the proportion of possible instances, or tokens, in which speakers use AAVE rather than SAE speech features. Random assignment to receive a housing voucher to move into a lower-poverty area (the intention-to-treat effect) led youth to live in neighborhoods (census tracts) with an 11 percentage point lower poverty rate on average over the next 10-15 y and reduced the share of AAVE tokens by ∼3 percentage points compared with the MTO control group youth. The MTO effect on AAVE use equals approximately half of the difference in AAVE frequency observed between youth whose parents have a high school diploma and those whose parents do not.

Morphology change and improved efficiency in organic photovoltaics via hexa-peri-hexabenzocoronene templates.

The morphology of the active layer in organic photovoltaics (OPVs) is of crucial importance as it greatly influences charge generation and transport. A templating interlayer between the electrode and the active layer can change active layer morphology and influence the domain orientation. A series of amphiphilic interface modifiers (IMs) combining a hydrophilic polyethylene-glycol (PEG) oligomer and a hydrophobic hexabenzocoronene (HBC) were designed to be soluble in PEDOT:PSS solutions, and surface accumulate on drying. These IMs are able to self-assemble in solution. When IMs are deposited on top of a poly(3,4-ethylenedioxythiophene) poly(styrenesulfonate) (PEDOT:PSS) film, they induce a morphology change of the active layer consisting of discotic fluorenyl-substituted HBC (FHBC) and [6,6]-phenyl C61-butyric acid methyl ester (PCBM). However, when only small amounts (0.2 wt %) of IMs are blended into PEDOT:PSS, a profound change of the active layer morphology is also observed. Morphology changes were monitored by grazing incidence wide-angle X-ray scattering (GIWAXS), transmission electron microscopy (TEM), TEM tomography, and low-energy high-angle angular dark-field scanning transmission electron microscopy (HAADF STEM). The interface modification resulted in a 20% enhancement of power conversion efficiency.

π-conjugated polymer-fullerene covalent hybrids via ambient conditions Diels-Alder ligation.

The established ability of graphitic carbon-nanomaterials to undergo ambient condition Diels-Alder reactions with cyclopentadienyl (Cp) groups is herein employed to prepare fullerene-polythiophene covalent hybrids with improved electron transfer and film forming characteristics. A novel precisely designed polythiophene (M n 9.8 kD, D 1.4) with 17 mol% of Cp-groups bearing repeat unit is prepared via Grignard metathesis polymerization. The UV/Vis absorption and fluorescence (λex 450 nm) characteristics of polythiophene with pendant Cp-groups (λmax 447 nm, λe-max 576 nm) are comparable to the reference poly(3-hexylthiophene) (λmax 450 nm, λe-max 576 nm). The novel polythiophene with pendant Cp-groups is capable of producing solvent-stable free-standing polythiophene films, and non-solvent assisted self-assemblies resulting in solvent-stable nanoporous-microstructures. (1) H-NMR spectroscopy reveals an efficient reaction of the pendant Cp-groups with C60 . The UV/Vis spectroscopic analyses of solution and thin films of the covalent and physical hybrids disclose closer donor-acceptor packing in the case of covalent hybrids. AFM images evidence that the covalent hybrids form smooth films with finer lamellar-organization. The effect is particularly remarkable in the case of poorly soluble C60 . A significant enhancement in photo-voltage is observed for all devices constituted of covalent hybrids, highlighting novel avenues to developing efficient electron donor-acceptor combinations for light harvesting systems.

Associations of housing mobility interventions for children in high-poverty neighborhoods with subsequent mental disorders during adolescence.

IMPORTANCE: Youth in high-poverty neighborhoods have high rates of emotional problems. Understanding neighborhood influences on mental health is crucial for designing neighborhood-level interventions. OBJECTIVE: To perform an exploratory analysis of associations between housing mobility interventions for children in high-poverty neighborhoods and subsequent mental disorders during adolescence. DESIGN, SETTING, AND PARTICIPANTS: The Moving to Opportunity Demonstration from 1994 to 1998 randomized 4604 volunteer public housing families with 3689 children in high-poverty neighborhoods into 1 of 2 housing mobility intervention groups (a low-poverty voucher group vs a traditional voucher group) or a control group. The low-poverty voucher group (n=1430) received vouchers to move to low-poverty neighborhoods with enhanced mobility counseling. The traditional voucher group (n=1081) received geographically unrestricted vouchers. Controls (n=1178) received no intervention. Follow-up evaluation was performed 10 to 15 years later (June 2008-April 2010) with participants aged 13 to 19 years (0-8 years at randomization). Response rates were 86.9% to 92.9%. MAIN OUTCOMES AND MEASURES: Presence of mental disorders from the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) within the past 12 months, including major depressive disorder, panic disorder, posttraumatic stress disorder (PTSD), oppositional-defiant disorder, intermittent explosive disorder, and conduct disorder, as assessed post hoc with a validated diagnostic interview. RESULTS: Of the 3689 adolescents randomized, 2872 were interviewed (1407 boys and 1465 girls). Compared with the control group, boys in the low-poverty voucher group had significantly increased rates of major depression (7.1% vs 3.5%; odds ratio (OR), 2.2 [95% CI, 1.2-3.9]), PTSD (6.2% vs 1.9%; OR, 3.4 [95% CI, 1.6-7.4]), and conduct disorder (6.4% vs 2.1%; OR, 3.1 [95% CI, 1.7-5.8]). Boys in the traditional voucher group had increased rates of PTSD compared with the control group (4.9% vs 1.9%, OR, 2.7 [95% CI, 1.2-5.8]). However, compared with the control group, girls in the traditional voucher group had decreased rates of major depression (6.5% vs 10.9%; OR, 0.6 [95% CI, 0.3-0.9]) and conduct disorder (0.3% vs 2.9%; OR, 0.1 [95% CI, 0.0-0.4]). CONCLUSIONS AND RELEVANCE: Interventions to encourage moving out of high-poverty neighborhoods were associated with increased rates of depression, PTSD, and conduct disorder among boys and reduced rates of depression and conduct disorder among girls. Better understanding of interactions among individual, family, and neighborhood risk factors is needed to guide future public housing policy changes.

Long-term effects of the Moving to Opportunity residential mobility experiment on crime and delinquency.

OBJECTIVES: Using data from a randomized experiment, to examine whether moving youth out of areas of concentrated poverty, where a disproportionate amount of crime occurs, prevents involvement in crime. METHODS: We draw on new administrative data from the U.S. Department of Housing and Urban Development's Moving to Opportunity (MTO) experiment. MTO families were randomized into an experimental group offered a housing voucher that could only be used to move to a low-poverty neighborhood, a Section 8 housing group offered a standard housing voucher, and a control group. This paper focuses on MTO youth ages 15-25 in 2001 (n = 4,643) and analyzes intention to treat effects on neighborhood characteristics and criminal behavior (number of violent- and property-crime arrests) through 10 years after randomization. RESULTS: We find the offer of a housing voucher generates large improvements in neighborhood conditions that attenuate over time and initially generates substantial reductions in violent-crime arrests and sizable increases in property-crime arrests for experimental group males. The crime effects attenuate over time along with differences in neighborhood conditions. CONCLUSIONS: Our findings suggest that criminal behavior is more strongly related to current neighborhood conditions (situational neighborhood effects) than to past neighborhood conditions (developmental neighborhood effects). The MTO design makes it difficult to determine which specific neighborhood characteristics are most important for criminal behavior. Our administrative data analyses could be affected by differences across areas in the likelihood that a crime results in an arrest.

The effects of housing and neighborhood conditions on child mortality.

In this paper we estimate the causal effects on child mortality from moving into less distressed neighborhood environments. We match mortality data covering the period from 1997 to 2009 with information on every child in public housing that applied for a housing voucher in Chicago in 1997 (N=11,680). Families were randomly assigned to the voucher wait list, and only some families were offered vouchers. The odds ratio for the effects of being offered a housing voucher on overall mortality rates is equal to 1.13 for all children (95% CI 0.73-1.70), 1.34 for boys (95% CI 0.85-2.05) and 0.21 for girls (95% CI 0.01-1.04).

Neighborhood effects on the long-term well-being of low-income adults.

Nearly 9 million Americans live in extreme-poverty neighborhoods, places that also tend to be racially segregated and dangerous. Yet, the effects on the well-being of residents of moving out of such communities into less distressed areas remain uncertain. Using data from Moving to Opportunity, a unique randomized housing mobility experiment, we found that moving from a high-poverty to lower-poverty neighborhood leads to long-term (10- to 15-year) improvements in adult physical and mental health and subjective well-being, despite not affecting economic self-sufficiency. A 1-standard deviation decline in neighborhood poverty (13 percentage points) increases subjective well-being by an amount equal to the gap in subjective well-being between people whose annual incomes differ by $13,000--a large amount given that the average control group income is $20,000. Subjective well-being is more strongly affected by changes in neighborhood economic disadvantage than racial segregation, which is important because racial segregation has been declining since 1970, but income segregation has been increasing.