Megastudy shows that reminders boost vaccination but adding free rides does not.

Encouraging routine COVID-19 vaccinations is likely to be a crucial policy challenge for decades to come. To avert hundreds of thousands of unnecessary hospitalizations and deaths, adoption will need to be higher than it was in the autumn of 2022 or 2023, when less than one-fifth of Americans received booster vaccines1,2. One approach to encouraging vaccination is to eliminate the friction of transportation hurdles. Previous research has shown that friction can hinder follow-through3 and that individuals who live farther from COVID-19 vaccination sites are less likely to get vaccinated4. However, the value of providing free round-trip transportation to vaccination sites is unknown. Here we show that offering people free round-trip Lyft rides to pharmacies has no benefit over and above sending them behaviourally informed text messages reminding them to get vaccinated. We determined this by running a megastudy with millions of CVS Pharmacy patients in the United States testing the effects of (1) free round-trip Lyft rides to CVS Pharmacies for vaccination appointments and (2) seven different sets of behaviourally informed vaccine reminder messages. Our results suggest that offering previously vaccinated individuals free rides to vaccination sites is not a good investment in the United States, contrary to the high expectations of both expert and lay forecasters. Instead, people in the United States should be sent behaviourally informed COVID-19 vaccination reminders, which increased the 30-day COVID-19 booster uptake by 21% (1.05 percentage points) and spilled over to increase 30-day influenza vaccinations by 8% (0.34 percentage points) in our megastudy. More rigorous testing of interventions to promote vaccination is needed to ensure that evidence-based solutions are deployed widely and that ineffective but intuitively appealing tools are discontinued.

Optimally generate policy-based evidence before scaling.

Social scientists have increasingly turned to the experimental method to understand human behaviour. One critical issue that makes solving social problems difficult is scaling up the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that, when we scale up ideas, most experience a 'voltage drop'-that is, on scaling, the cost-benefit profile depreciates considerably. Here I argue that, to reduce voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information should be generated and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations in which I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to have scaling confidence?

The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs.

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Shifting parental beliefs about child development to foster parental investments and improve school readiness outcomes.

Socioeconomic gaps in child development open up early, with associated disparities in parental investments in children. Understanding the drivers of these disparities is key to designing effective policies. We first show that parental beliefs about the impact of early parental investments differ across socioeconomic status (SES), with parents of higher SES being more likely to believe that parental investments impact child development. We then use two randomized controlled trials to explore the mutability of such beliefs and their link to parental investments and child development, our three primary outcomes. In the first trial (NCT02812017 on clinicaltrials.gov), parents in the treatment group were asked to watch a short educational video during four well-child visits with their pediatrician while in the second trial (NCT03076268), parents in the treatment group received twelve home visits with feedback based on their daily interactions with their child. In both cases, we find that parental beliefs about child development are malleable. The first program changes parental beliefs but fails to lastingly increase parental investments and child outcomes. By contrast, in the more intensive program, all pre-specified endpoints are improved: the augmented beliefs are associated with enriched parent-child interactions and higher vocabulary, math, and social-emotional skills for the children.

Do Financial Incentives Aimed at Decreasing Interhousehold Inequality Increase Intrahousehold Inequality?

Research has shown that giving disadvantaged families financial incentives to invest in their children could decrease socioeconomic inequality by enhancing human capital formation. Yet, within the household how are such gains achieved? We use a field experiment to investigate how parents allocate time when they receive financial incentives. We find that incentives increase investment in the target child. But, parents achieve these gains by substituting away from time spent with the child's sibling(s). An unintended consequence is that intrahousehold inequality increases and aggregate gains from the program are overstated when focusing only on target children.

An experimental test of fundraising appeals targeting donor and recipient benefits.

We partnered with Alaska's Pick.Click.Give. programme to implement a statewide natural field experiment with 540,000 Alaskans designed to examine two of the main motivations for charitable giving: concerns for the benefits to self (impure altruism or 'warm glow') or concerns for the benefits to others (pure altruism). Our empirical results highlight the relative importance of appeals to self: individuals who received such an appeal were 6.6% more likely to give and gave 23% more than counterparts in the control group. Yet, a message that instead appealed to recipient benefits (motivated by altruism) had no statistically significant effect on average donations relative to the control group. We also find evidence of long-run effects of warm-glow appeals in the subsequent year. Our results have import for theoreticians and empiricists interested in modelling charitable giving as well as practitioners and policymakers.

How can Bill and Melinda Gates increase other people's donations to fund public goods?

We conducted a fundraising experiment with an international development nonprofit organization in which a matching grant offered by the Bill and Melinda Gates Foundation raised more funds than one from an anonymous donor. The effect is strongest for solicitees who previously gave to other BMGF-supported, poverty charities. With supporting evidence from two other fundraising experiments as well as a survey experiment, we argue this is consistent with a quality signal mechanism. Alternative mechanisms are discussed, and not ruled out. The results help inform theories about charitable giving decision-making, and provide guidance to organizations and large donors on how to overcome information asymmetries hindering fundraising.

Risk Preferences of Children and Adolescents in Relation to Gender, Cognitive Skills, Soft Skills, and Executive Functions.

We conduct experiments eliciting risk preferences with over 1,400 children and adolescents aged 3-15 years old. We complement our data with an assessment of cognitive and executive function skills. First, we find that adolescent girls display significantly greater risk aversion than adolescent boys. This pattern is not observed among young children, suggesting that the gender gap in risk preferences emerges in early adolescence. Second, we find that at all ages in our study, cognitive skills (specifically math ability) are positively associated with risk taking. Executive functions among children, and soft skills among adolescents, are negatively associated with risk taking. Third, we find that greater risk-tolerance is associated with higher likelihood of disciplinary referrals, which provides evidence that our task is equipped to measure a relevant behavioral outcome. For academics, our research provides a deeper understanding of the developmental origins of risk preferences and highlights the important role of cognitive and executive function skills to better understand the association between risk preferences and cognitive abilities over the studied age range.

The Effect of Early Childhood Education on Social Preferences.

We present results from the first study to examine the causal impact of early childhood education on the social preferences of children. We compare children who, at 3-4 years old, were randomized into either a full-time preschool, a parenting program, or a control group. We returned to the children when they reached 6-8 years of age and conducted a series of incentivized experiments to elicit their social preferences. We find that early childhood education has a strong causal impact on social preferences. Our findings highlight the importance of taking a broad perspective when designing and evaluating early childhood educational programs.

How natural field experiments have enhanced our understanding of unemployment.

Natural field experiments investigating key labour market phenomena such as unemployment have only been used since the early 2000s. This paper reviews the literature and draws three primary conclusions that deepen our understanding of unemployment. First, the inability to monitor workers perfectly in many occupations complicates the hiring decision in a way that contributes to unemployment. Second, the inability to determine a worker's attributes precisely at the time of hiring leads to discrimination on the basis of factors such as race, gender, age and ethnicity. This can lead to systematically high and persistent levels of unemployment for groups that face discrimination. Third, the importance of social and personal dynamics in the workplace can lead to short-term unemployment. Much of the knowledge necessary for these conclusions could only be obtained using natural field experiments due to their ability to combine randomized control with an absence of experimenter demand effects.

Toward an understanding of the development of time preferences: Evidence from field experiments.

Time preferences have been correlated with a range of life outcomes, yet little is known about their early development. We conduct a field experiment to elicit time preferences of over 1,200 children ages 3-12, who make several intertemporal decisions. To shed light on how such primitives form, we explore various channels that might affect time preferences, from background characteristics to the causal impact of an early schooling program that we developed and operated. Our results suggest that time preferences evolve substantially during this period, with younger children displaying more impatience than older children. We also find a strong association with race: black children, relative to white or Hispanic children, are more impatient. Finally, assignment to different schooling opportunities is not significantly associated with child time preferences.

Testing for crowd out in social nudges: Evidence from a natural field experiment in the market for electricity.

This study considers the response of household electricity consumption to social nudges during peak load events. Our investigation considers two social nudges. The first targets conservation during peak load events, while the second promotes aggregate conservation. Using data from a natural field experiment with 42,100 households, we find that both social nudges reduce peak load electricity consumption by 2 to 4% when implemented in isolation and by nearly 7% when implemented in combination. These findings suggest an important role for social nudges in the regulation of electricity markets and a limited role for crowd out effects.

The test-retest reliability of the latent construct of executive function depends on whether tasks are represented as formative or reflective indicators.

This study investigates the test-retest reliability of a battery of executive function (EF) tasks with a specific interest in testing whether the method that is used to create a battery-wide score would result in differences in the apparent test-retest reliability of children's performance. A total of 188 4-year-olds completed a battery of computerized EF tasks twice across a period of approximately two weeks. Two different approaches were used to create a score that indexed children's overall performance on the battery-i.e., (1) the mean score of all completed tasks and (2) a factor score estimate which used confirmatory factor analysis (CFA). Pearson and intra-class correlations were used to investigate the test-retest reliability of individual EF tasks, as well as an overall battery score. Consistent with previous studies, the test-retest reliability of individual tasks was modest (rs ≈ .60). The test-retest reliability of the overall battery scores differed depending on the scoring approach (rmean = .72; rfactor_score = .99). It is concluded that the children's performance on individual EF tasks exhibit modest levels of test-retest reliability. This underscores the importance of administering multiple tasks and aggregating performance across these tasks in order to improve precision of measurement. However, the specific strategy that is used has a large impact on the apparent test-retest reliability of the overall score. These results replicate our earlier findings and provide additional cautionary evidence against the routine use of factor analytic approaches for representing individual performance across a battery of EF tasks.

Trading experience modulates anterior insula to reduce the endowment effect.

People often demand a greater price when selling goods that they own than they would pay to purchase the same goods-a well-known economic bias called the endowment effect. The endowment effect has been found to be muted among experienced traders, but little is known about how trading experience reduces the endowment effect. We show that when selling, experienced traders exhibit lower right anterior insula activity, but no differences in nucleus accumbens or orbitofrontal activation, compared with inexperienced traders. Furthermore, insula activation mediates the effect of experience on the endowment effect. Similar results are obtained for inexperienced traders who are incentivized to gain trading experience. This finding indicates that frequent trading likely mitigates the endowment effect indirectly by modifying negative affective responses in the context of selling.