Partisan Stability During Turbulent Times: Evidence from Three American Panel Surveys.

The past decade has witnessed profound changes in the tenor of American party politics. These changes, in tandem with growing affective polarization and residential segregation by party, raise the question of whether party identification is itself changing. Using three multi-wave panel surveys that stretch from the first Obama Administration through the Trump Administration, this paper takes a fresh look at the stability of party identification, using several different statistical approaches to differentiate true partisan change from response error. Perhaps surprisingly, the pace of partisan change observed between 2011 and 2020 is quite similar to the apparent rates of change in panel surveys dating back to the 1950s. Few respondents experience appreciable change in party identification in the short run, but the pace at which partisanship changes implies that substantial changes are relatively common over a voter's lifespan. Supplementary Information: The online version contains supplementary material available at 10.1007/s11109-022-09825-y.

A Radio Drama's Effects on HIV Attitudes and Policy Priorities: A Field Experiment in Tanzania.

A growing body of evidence investigates how entertainment education influences knowledge about HIV, stigma toward those with HIV, and openness to disclosing one's HIV status. The present study shows that in addition to these effects, mass media interventions may influence audiences' policy priorities, such as their demand for local access to HIV/AIDS medical care. A condensed (2 hours) version of a popular Swahili radio drama was presented to rural Tanzanians as part of a placebo-controlled experiment, clustered at the village level. A random sample comprising 1,200 participants were interviewed at baseline and invited to attend a presentation of the radio drama, and 83% attended. Baseline respondents were reinterviewed 2 weeks later with a response rate of 95%. In addition to increasing listeners' knowledge and support for disclosure of HIV status, the radio drama produced sizable and statistically significant effects on listeners' preference for hypothetical candidates promising improved HIV/AIDS treatment.

Prejudice Reduction: Progress and Challenges.

The past decade has seen rapid growth in research that evaluates methods for reducing prejudice. This essay reviews 418 experiments reported in 309 manuscripts from 2007 to 2019 to assess which approaches work best and why. Our quantitative assessment uses meta-analysis to estimate average effects. Our qualitative assessment calls attention to landmark studies that are noteworthy for sustained interventions, imaginative measurement, and transparency. However, 76% of all studies evaluate light touch interventions, the long-term impact of which remains unclear. The modal intervention uses mentalizing as a salve for prejudice. Although these studies report optimistic conclusions, we identify troubling indications of publication bias that may exaggerate effects. Furthermore, landmark studies often find limited effects, which suggests the need for further theoretical innovation or synergies with other kinds of psychological or structural interventions. We conclude that much research effort is theoretically and empirically ill-suited to provide actionable, evidence-based recommendations for reducing prejudice.

Optimal Allocation of Interviews to Baseline and Endline Surveys in Place-Based Randomized Trials and Quasi-Experiments.

BACKGROUND: Many place-based randomized trials and quasi-experiments use a pair of cross-section surveys, rather than panel surveys, to estimate the average treatment effect of an intervention. In these studies, a random sample of individuals in each geographic cluster is selected for a baseline (preintervention) survey, and an independent random sample is selected for an endline (postintervention) survey. OBJECTIVE: This design raises the question, given a fixed budget, how should a researcher allocate resources between the baseline and endline surveys to maximize the precision of the estimated average treatment effect? RESULTS: We formalize this allocation problem and show that although the optimal share of interviews allocated to the baseline survey is always less than one-half, it is an increasing function of the total number of interviews per cluster, the cluster-level correlation between the baseline measure and the endline outcome, and the intracluster correlation coefficient. An example using multicountry survey data from Africa illustrates how the optimal allocation formulas can be combined with data to inform decisions at the planning stage. Another example uses data from a digital political advertising experiment in Texas to explore how precision would have varied with alternative allocations.

Assessing methods for generalizing experimental impact estimates to target populations.

Randomized experiments are considered the gold standard for causal inference, as they can provide unbiased estimates of treatment effects for the experimental participants. However, researchers and policymakers are often interested in using a specific experiment to inform decisions about other target populations. In education research, increasing attention is being paid to the potential lack of generalizability of randomized experiments, as the experimental participants may be unrepresentative of the target population of interest. This paper examines whether generalization may be assisted by statistical methods that adjust for observed differences between the experimental participants and members of a target population. The methods examined include approaches that reweight the experimental data so that participants more closely resemble the target population and methods that utilize models of the outcome. Two simulation studies and one empirical analysis investigate and compare the methods' performance. One simulation uses purely simulated data while the other utilizes data from an evaluation of a school-based dropout prevention program. Our simulations suggest that machine learning methods outperform regression-based methods when the required structural (ignorability) assumptions are satisfied. When these assumptions are violated, all of the methods examined perform poorly. Our empirical analysis uses data from a multi-site experiment to assess how well results from a given site predict impacts in other sites. Using a variety of extrapolation methods, predicted effects for each site are compared to actual benchmarks. Flexible modeling approaches perform best, although linear regression is not far behind. Taken together, these results suggest that flexible modeling techniques can aid generalization while underscoring the fact that even state-of-the-art statistical techniques still rely on strong assumptions.

Does Product Placement Change Television Viewers' Social Behavior?

To what extent are television viewers affected by the behaviors and decisions they see modeled by characters in television soap operas? Collaborating with scriptwriters for three prime-time nationally-broadcast Spanish-language telenovelas, we embedded scenes about topics such as drunk driving or saving money at randomly assigned periods during the broadcast season. Outcomes were measured unobtrusively by aggregate city- and nation-wide time series, such as the number of Hispanic motorists arrested daily for drunk driving or the number of accounts opened in banks located in Hispanic neighborhoods. Results indicate that while two of the treatment effects are statistically significant, none are substantively large or long-lasting. Actions that could be taken during the immediate viewing session, like online searching, and those that were relatively more integrated into the telenovela storyline, specifically reducing cholesterol, were briefly affected, but not behaviors requiring sustained efforts, like opening a bank account or registering to vote.

Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence.

Survey respondents may give untruthful answers to sensitive questions when asked directly. In recent years, researchers have turned to the list experiment (also known as the item count technique) to overcome this difficulty. While list experiments are arguably less prone to bias than direct questioning, list experiments are also more susceptible to sampling variability. We show that researchers need not abandon direct questioning altogether in order to gain the advantages of list experimentation. We develop a nonparametric estimator of the prevalence of sensitive behaviors that combines list experimentation and direct questioning. We prove that this estimator is asymptotically more efficient than the standard difference-in-means estimator, and we provide a basis for inference using Wald-type confidence intervals. Additionally, leveraging information from the direct questioning, we derive two nonparametric placebo tests for assessing identifying assumptions underlying list experiments. We demonstrate the effectiveness of our combined estimator and placebo tests with an original survey experiment.

Political science. When contact changes minds: an experiment on transmission of support for gay equality.

Can a single conversation change minds on divisive social issues, such as same-sex marriage? A randomized placebo-controlled trial assessed whether gay (n = 22) or straight (n = 19) messengers were effective at encouraging voters (n = 972) to support same-sex marriage and whether attitude change persisted and spread to others in voters' social networks. The results, measured by an unrelated panel survey, show that both gay and straight canvassers produced large effects initially, but only gay canvassers' effects persisted in 3-week, 6-week, and 9-month follow-ups. We also find strong evidence of within-household transmission of opinion change, but only in the wake of conversations with gay canvassers. Contact with gay canvassers further caused substantial change in the ratings of gay men and lesbians more generally. These large, persistent, and contagious effects were confirmed by a follow-up experiment. Contact with minorities coupled with discussion of issues pertinent to them is capable of producing a cascade of opinion change.

Yes, but what's the mechanism? (don't expect an easy answer).

Psychologists increasingly recommend experimental analysis of mediation. This is a step in the right direction because mediation analyses based on nonexperimental data are likely to be biased and because experiments, in principle, provide a sound basis for causal inference. But even experiments cannot overcome certain threats to inference that arise chiefly or exclusively in the context of mediation analysis-threats that have received little attention in psychology. The authors describe 3 of these threats and suggest ways to improve the exposition and design of mediation tests. Their conclusion is that inference about mediators is far more difficult than previous research suggests and is best tackled by an experimental research program that is specifically designed to address the challenges of mediation analysis.

Prejudice reduction: what works? A review and assessment of research and practice.

This article reviews the observational, laboratory, and field experimental literatures on interventions for reducing prejudice. Our review places special emphasis on assessing the methodological rigor of existing research, calling attention to problems of design and measurement that threaten both internal and external validity. Of the hundreds of studies we examine, a small fraction speak convincingly to the questions of whether, why, and under what conditions a given type of intervention works. We conclude that the causal effects of many widespread prejudice-reduction interventions, such as workplace diversity training and media campaigns, remain unknown. Although some intergroup contact and cooperation interventions appear promising, a much more rigorous and broad-ranging empirical assessment of prejudice-reduction strategies is needed to determine what works.