Optimally generate policy-based evidence before scaling.

Social scientists have increasingly turned to the experimental method to understand human behaviour. One critical issue that makes solving social problems difficult is scaling up the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that, when we scale up ideas, most experience a 'voltage drop'-that is, on scaling, the cost-benefit profile depreciates considerably. Here I argue that, to reduce voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information should be generated and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations in which I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to have scaling confidence?

Megastudy shows that reminders boost vaccination but adding free rides does not.

Encouraging routine COVID-19 vaccinations is likely to be a crucial policy challenge for decades to come. To avert hundreds of thousands of unnecessary hospitalizations and deaths, adoption will need to be higher than it was in the autumn of 2022 or 2023, when less than one-fifth of Americans received booster vaccines1,2. One approach to encouraging vaccination is to eliminate the friction of transportation hurdles. Previous research has shown that friction can hinder follow-through3 and that individuals who live farther from COVID-19 vaccination sites are less likely to get vaccinated4. However, the value of providing free round-trip transportation to vaccination sites is unknown. Here we show that offering people free round-trip Lyft rides to pharmacies has no benefit over and above sending them behaviourally informed text messages reminding them to get vaccinated. We determined this by running a megastudy with millions of CVS Pharmacy patients in the United States testing the effects of (1) free round-trip Lyft rides to CVS Pharmacies for vaccination appointments and (2) seven different sets of behaviourally informed vaccine reminder messages. Our results suggest that offering previously vaccinated individuals free rides to vaccination sites is not a good investment in the United States, contrary to the high expectations of both expert and lay forecasters. Instead, people in the United States should be sent behaviourally informed COVID-19 vaccination reminders, which increased the 30-day COVID-19 booster uptake by 21% (1.05 percentage points) and spilled over to increase 30-day influenza vaccinations by 8% (0.34 percentage points) in our megastudy. More rigorous testing of interventions to promote vaccination is needed to ensure that evidence-based solutions are deployed widely and that ineffective but intuitively appealing tools are discontinued.

Testing for crowd out in social nudges: Evidence from a natural field experiment in the market for electricity.

This study considers the response of household electricity consumption to social nudges during peak load events. Our investigation considers two social nudges. The first targets conservation during peak load events, while the second promotes aggregate conservation. Using data from a natural field experiment with 42,100 households, we find that both social nudges reduce peak load electricity consumption by 2 to 4% when implemented in isolation and by nearly 7% when implemented in combination. These findings suggest an important role for social nudges in the regulation of electricity markets and a limited role for crowd out effects.

The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs.

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Do Financial Incentives Aimed at Decreasing Interhousehold Inequality Increase Intrahousehold Inequality?

Research has shown that giving disadvantaged families financial incentives to invest in their children could decrease socioeconomic inequality by enhancing human capital formation. Yet, within the household how are such gains achieved? We use a field experiment to investigate how parents allocate time when they receive financial incentives. We find that incentives increase investment in the target child. But, parents achieve these gains by substituting away from time spent with the child's sibling(s). An unintended consequence is that intrahousehold inequality increases and aggregate gains from the program are overstated when focusing only on target children.

How can Bill and Melinda Gates increase other people's donations to fund public goods?

We conducted a fundraising experiment with an international development nonprofit organization in which a matching grant offered by the Bill and Melinda Gates Foundation raised more funds than one from an anonymous donor. The effect is strongest for solicitees who previously gave to other BMGF-supported, poverty charities. With supporting evidence from two other fundraising experiments as well as a survey experiment, we argue this is consistent with a quality signal mechanism. Alternative mechanisms are discussed, and not ruled out. The results help inform theories about charitable giving decision-making, and provide guidance to organizations and large donors on how to overcome information asymmetries hindering fundraising.

Risk Preferences of Children and Adolescents in Relation to Gender, Cognitive Skills, Soft Skills, and Executive Functions.

We conduct experiments eliciting risk preferences with over 1,400 children and adolescents aged 3-15 years old. We complement our data with an assessment of cognitive and executive function skills. First, we find that adolescent girls display significantly greater risk aversion than adolescent boys. This pattern is not observed among young children, suggesting that the gender gap in risk preferences emerges in early adolescence. Second, we find that at all ages in our study, cognitive skills (specifically math ability) are positively associated with risk taking. Executive functions among children, and soft skills among adolescents, are negatively associated with risk taking. Third, we find that greater risk-tolerance is associated with higher likelihood of disciplinary referrals, which provides evidence that our task is equipped to measure a relevant behavioral outcome. For academics, our research provides a deeper understanding of the developmental origins of risk preferences and highlights the important role of cognitive and executive function skills to better understand the association between risk preferences and cognitive abilities over the studied age range.

Quantity discounts on a virtual good: The results of a massive pricing experiment at King Digital Entertainment.

We report on a natural field experiment on quantity discounts involving more than 14 million consumers. Implementing price reductions ranging from 9-70% for large purchases, we found remarkably little impact on revenue, either positively or negatively. There was virtually no increase in the quantity of customers making a purchase; all the observed changes occurred for customers who already were buyers. We found evidence that infrequent purchasers are more responsive to discounts than frequent purchasers. There was some evidence of habit formation when prices returned to pre-experiment levels. There also was some evidence that consumers contemplating small purchases are discouraged by the presence of extreme quantity discounts for large purchases.

Trading experience modulates anterior insula to reduce the endowment effect.

People often demand a greater price when selling goods that they own than they would pay to purchase the same goods-a well-known economic bias called the endowment effect. The endowment effect has been found to be muted among experienced traders, but little is known about how trading experience reduces the endowment effect. We show that when selling, experienced traders exhibit lower right anterior insula activity, but no differences in nucleus accumbens or orbitofrontal activation, compared with inexperienced traders. Furthermore, insula activation mediates the effect of experience on the endowment effect. Similar results are obtained for inexperienced traders who are incentivized to gain trading experience. This finding indicates that frequent trading likely mitigates the endowment effect indirectly by modifying negative affective responses in the context of selling.

Toward an understanding of the development of time preferences: Evidence from field experiments.

Time preferences have been correlated with a range of life outcomes, yet little is known about their early development. We conduct a field experiment to elicit time preferences of over 1,200 children ages 3-12, who make several intertemporal decisions. To shed light on how such primitives form, we explore various channels that might affect time preferences, from background characteristics to the causal impact of an early schooling program that we developed and operated. Our results suggest that time preferences evolve substantially during this period, with younger children displaying more impatience than older children. We also find a strong association with race: black children, relative to white or Hispanic children, are more impatient. Finally, assignment to different schooling opportunities is not significantly associated with child time preferences.

Rise and fall of competitiveness in individualistic and collectivistic societies.

Competitiveness pervades life: plants compete for sunlight and water, animals for territory and food, and humans for mates and income. Herein we investigate human competitiveness with a natural experiment and a set of behavioral experiments. We compare competitiveness in traditional fishing societies where local natural forces determine whether fishermen work in isolation or in collectives. We find sharp evidence that fishermen from individualistic societies are far more competitive than fishermen from collectivistic societies, and that this difference emerges with work experience. These findings suggest that humans can evolve traits to specific needs, support the idea that socio-ecological factors play a decisive role for individual competitiveness, and provide evidence how individualistic and collectivistic societies shape economic behavior.

Nurture affects gender differences in spatial abilities.

Women remain significantly underrepresented in the science, engineering, and technology workforce. Some have argued that spatial ability differences, which represent the most persistent gender differences in the cognitive literature, are partly responsible for this gap(.) The underlying forces at work shaping the observed spatial ability differences revolve naturally around the relative roles of nature and nurture. Although these forces remain among the most hotly debated in all of the sciences, the evidence for nurture is tenuous, because it is difficult to compare gender differences among biologically similar groups with distinct nurture. In this study, we use a large-scale incentivized experiment with nearly 1,300 participants to show that the gender gap in spatial abilities, measured by time to solve a puzzle, disappears when we move from a patrilineal society to an adjoining matrilineal society. We also show that about one-third of the effect can be explained by differences in education. Given that none of our participants have experience with puzzle solving and that villagers from both societies have the same means of subsistence and shared genetic background, we argue that these results show the role of nurture in the gender gap in cognitive abilities.

Critical thinking and misinformation vulnerability: experimental evidence from Colombia.

Misinformation represents a vital threat to the societal fabric of modern economies. While skills interventions to detect misinformation such as de-bunking and prebunking, media literacy, and manipulation resilience have begun to receive increased attention, evidence on de-biasing interventions and their link with misinformation vulnerability is scarce. We explore the demand for misinformation through the lens of augmenting critical thinking in an online framed field experiment during the 2022 Presidential election in Colombia. Data from roughly 2.000 individuals suggest that providing individuals with information about their own biases (obtained through a personality test) has no impact on skepticism towards news. But (additionally) showing participants a de-biasing video seems to enhance critical thinking, causing subjects to more carefully consider the truthfulness of potential misinformation.

Shifting parental beliefs about child development to foster parental investments and improve school readiness outcomes.

Socioeconomic gaps in child development open up early, with associated disparities in parental investments in children. Understanding the drivers of these disparities is key to designing effective policies. We first show that parental beliefs about the impact of early parental investments differ across socioeconomic status (SES), with parents of higher SES being more likely to believe that parental investments impact child development. We then use two randomized controlled trials to explore the mutability of such beliefs and their link to parental investments and child development, our three primary outcomes. In the first trial (NCT02812017 on clinicaltrials.gov), parents in the treatment group were asked to watch a short educational video during four well-child visits with their pediatrician while in the second trial (NCT03076268), parents in the treatment group received twelve home visits with feedback based on their daily interactions with their child. In both cases, we find that parental beliefs about child development are malleable. The first program changes parental beliefs but fails to lastingly increase parental investments and child outcomes. By contrast, in the more intensive program, all pre-specified endpoints are improved: the augmented beliefs are associated with enriched parent-child interactions and higher vocabulary, math, and social-emotional skills for the children.

An experimental test of fundraising appeals targeting donor and recipient benefits.

We partnered with Alaska's Pick.Click.Give. programme to implement a statewide natural field experiment with 540,000 Alaskans designed to examine two of the main motivations for charitable giving: concerns for the benefits to self (impure altruism or 'warm glow') or concerns for the benefits to others (pure altruism). Our empirical results highlight the relative importance of appeals to self: individuals who received such an appeal were 6.6% more likely to give and gave 23% more than counterparts in the control group. Yet, a message that instead appealed to recipient benefits (motivated by altruism) had no statistically significant effect on average donations relative to the control group. We also find evidence of long-run effects of warm-glow appeals in the subsequent year. Our results have import for theoreticians and empiricists interested in modelling charitable giving as well as practitioners and policymakers.

The Effect of Early Childhood Education on Social Preferences.

We present results from the first study to examine the causal impact of early childhood education on the social preferences of children. We compare children who, at 3-4 years old, were randomized into either a full-time preschool, a parenting program, or a control group. We returned to the children when they reached 6-8 years of age and conducted a series of incentivized experiments to elicit their social preferences. We find that early childhood education has a strong causal impact on social preferences. Our findings highlight the importance of taking a broad perspective when designing and evaluating early childhood educational programs.

How natural field experiments have enhanced our understanding of unemployment.

Natural field experiments investigating key labour market phenomena such as unemployment have only been used since the early 2000s. This paper reviews the literature and draws three primary conclusions that deepen our understanding of unemployment. First, the inability to monitor workers perfectly in many occupations complicates the hiring decision in a way that contributes to unemployment. Second, the inability to determine a worker's attributes precisely at the time of hiring leads to discrimination on the basis of factors such as race, gender, age and ethnicity. This can lead to systematically high and persistent levels of unemployment for groups that face discrimination. Third, the importance of social and personal dynamics in the workplace can lead to short-term unemployment. Much of the knowledge necessary for these conclusions could only be obtained using natural field experiments due to their ability to combine randomized control with an absence of experimenter demand effects.