Integrating explanation and prediction in computational social science.

Computational social science is more than just large repositories of digital data and the computational methods needed to construct and analyse them. It also represents a convergence of different fields with different ways of thinking about and doing science. The goal of this Perspective is to provide some clarity around how these approaches differ from one another and to propose how they might be productively integrated. Towards this end we make two contributions. The first is a schema for thinking about research activities along two dimensions-the extent to which work is explanatory, focusing on identifying and estimating causal effects, and the degree of consideration given to testing predictions of outcomes-and how these two priorities can complement, rather than compete with, one another. Our second contribution is to advocate that computational social scientists devote more attention to combining prediction and explanation, which we call integrative modelling, and to outline some practical suggestions for realizing this goal.

Megastudies improve the impact of applied behavioural science.

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens' decisions and outcomes1. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals2. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy-a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research3-6. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies.

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was "waiting for you." Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

Algorithms as discrimination detectors.

Preventing discrimination requires that we have means of detecting it, and this can be enormously difficult when human beings are making the underlying decisions. As applied today, algorithms can increase the risk of discrimination. But as we argue here, algorithms by their nature require a far greater level of specificity than is usually possible with human decision making, and this specificity makes it possible to probe aspects of the decision in additional ways. With the right changes to legal and regulatory systems, algorithms can thus potentially make it easier to detect-and hence to help prevent-discrimination.

Scarcity frames value.

Economic models of decision making assume that people have a stable way of thinking about value. In contrast, psychology has shown that people's preferences are often malleable and influenced by normatively irrelevant contextual features. Whereas economics derives its predictions from the assumption that people navigate a world of scarce resources, recent psychological work has shown that people often do not attend to scarcity. In this article, we show that when scarcity does influence cognition, it renders people less susceptible to classic context effects. Under conditions of scarcity, people focus on pressing needs and recognize the trade-offs that must be made against those needs. Those trade-offs frame perception more consistently than irrelevant contextual cues, which exert less influence. The results suggest that scarcity can align certain behaviors more closely with traditional economic predictions.

The Inversion Problem: Why Algorithms Should Infer Mental State and Not Just Predict Behavior.

More and more machine learning is applied to human behavior. Increasingly these algorithms suffer from a hidden-but serious-problem. It arises because they often predict one thing while hoping for another. Take a recommender system: It predicts clicks but hopes to identify preferences. Or take an algorithm that automates a radiologist: It predicts in-the-moment diagnoses while hoping to identify their reflective judgments. Psychology shows us the gaps between the objectives of such prediction tasks and the goals we hope to achieve: People can click mindlessly; experts can get tired and make systematic errors. We argue such situations are ubiquitous and call them "inversion problems": The real goal requires understanding a mental state that is not directly measured in behavioral data but must instead be inverted from the behavior. Identifying and solving these problems require new tools that draw on both behavioral and computational science.

Health insurance coverage and take-up: lessons from behavioral economics.

CONTEXT: Millions of uninsured Americans ostensibly have insurance available to them-many at very low cost-but do not take it up. Traditional economic analysis is based on the premise that these are rational decisions, but it is hard to reconcile observed enrollment patterns with this view. The policy prescriptions that the traditional model generates may thus fail to achieve their goals. Behavioral economics, which integrates insights from psychology into economic analysis, identifies important deviations from the traditional assumptions of rationality and can thus improve our understanding of what drives health insurance take-up and improved policy design. METHODS: Rather than a systematic review of the coverage literature, this article is a primer for considering issues in health insurance coverage from a behavioral economics perspective, supplementing the standard model. We present relevant evidence on decision making and insurance take-up and use it to develop a behavioral approach to both the policy problem posed by the lack of health insurance coverage and possible policy solutions to that problem. FINDINGS: We found that evidence from behavioral economics can shed light on both the sources of low take-up and the efficacy of different policy levers intended to expand coverage. We then applied these insights to policy design questions for public and private insurance coverage and to the implementation of the recently enacted health reform, focusing on the use of behavioral insights to maximize the value of spending on coverage. CONCLUSIONS: We concluded that the success of health insurance coverage reform depends crucially on understanding the behavioral barriers to take-up. The take-up process is likely governed by psychology as much as economics, and public resources can likely be used much more effectively with behaviorally informed policy design.

An algorithmic approach to reducing unexplained pain disparities in underserved populations.

Underserved populations experience higher levels of pain. These disparities persist even after controlling for the objective severity of diseases like osteoarthritis, as graded by human physicians using medical images, raising the possibility that underserved patients' pain stems from factors external to the knee, such as stress. Here we use a deep learning approach to measure the severity of osteoarthritis, by using knee X-rays to predict patients' experienced pain. We show that this approach dramatically reduces unexplained racial disparities in pain. Relative to standard measures of severity graded by radiologists, which accounted for only 9% (95% confidence interval (CI), 3-16%) of racial disparities in pain, algorithmic predictions accounted for 43% of disparities, or 4.7× more (95% CI, 3.2-11.8×), with similar results for lower-income and less-educated patients. This suggests that much of underserved patients' pain stems from factors within the knee not reflected in standard radiographic measures of severity. We show that the algorithm's ability to reduce unexplained disparities is rooted in the racial and socioeconomic diversity of the training set. Because algorithmic severity measures better capture underserved patients' pain, and severity measures influence treatment decisions, algorithmic predictions could potentially redress disparities in access to treatments like arthroplasty.

A Comparison of Patient History- and EKG-based Cardiac Risk Scores.

Patient-specific risk scores are used to identify individuals at elevated risk for cardiovascular disease. Typically, risk scores are based on patient habits and medical history - age, sex, race, smoking behavior, and prior vital signs and diagnoses. We explore an alternative source of information, a patient's raw electrocardiogram recording, and develop a score of patient risk for various outcomes. We compare models that predict adverse cardiac outcomes following an emergency department visit, and show that a learned representation (e.g. deep neural network) of raw EKG waveforms can improve prediction over traditional risk factors. Further, we show that a simple model based on segmented heart beats performs as well or better than a complex convolutional network recently shown to reliably automate arrhythmia detection in EKGs. We analyze a large cohort of emergency department patients and show evidence that EKG-derived scores can be more robust to patient heterogeneity.

Dissecting racial bias in an algorithm used to manage the health of populations.

Health systems rely on commercial prediction algorithms to identify and help patients with complex health needs. We show that a widely used algorithm, typical of this industry-wide approach and affecting millions of patients, exhibits significant racial bias: At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses. Remedying this disparity would increase the percentage of Black patients receiving additional help from 17.7 to 46.5%. The bias arises because the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. We suggest that the choice of convenient, seemingly effective proxies for ground truth can be an important source of algorithmic bias in many contexts.

Predictive modeling of U.S. health care spending in late life.

That one-quarter of Medicare spending in the United States occurs in the last year of life is commonly interpreted as waste. But this interpretation presumes knowledge of who will die and when. Here we analyze how spending is distributed by predicted mortality, based on a machine-learning model of annual mortality risk built using Medicare claims. Death is highly unpredictable. Less than 5% of spending is accounted for by individuals with predicted mortality above 50%. The simple fact that we spend more on the sick-both on those who recover and those who die-accounts for 30 to 50% of the concentration of spending on the dead. Our results suggest that spending on the ex post dead does not necessarily mean that we spend on the ex ante "hopeless."

HUMAN DECISIONS AND MACHINE PREDICTIONS.

Can machine learning improve human decision making? Bail decisions provide a good test case. Millions of times each year, judges make jail-or-release decisions that hinge on a prediction of what a defendant would do if released. The concreteness of the prediction task combined with the volume of data available makes this a promising machine-learning application. Yet comparing the algorithm to judges proves complicated. First, the available data are generated by prior judge decisions. We only observe crime outcomes for released defendants, not for those judges detained. This makes it hard to evaluate counterfactual decision rules based on algorithmic predictions. Second, judges may have a broader set of preferences than the variable the algorithm predicts; for instance, judges may care specifically about violent crimes or about racial inequities. We deal with these problems using different econometric strategies, such as quasi-random assignment of cases to judges. Even accounting for these concerns, our results suggest potentially large welfare gains: one policy simulation shows crime reductions up to 24.7% with no change in jailing rates, or jailing rate reductions up to 41.9% with no increase in crime rates. Moreover, all categories of crime, including violent crimes, show reductions; and these gains can be achieved while simultaneously reducing racial disparities. These results suggest that while machine learning can be valuable, realizing this value requires integrating these tools into an economic framework: being clear about the link between predictions and decisions; specifying the scope of payoff functions; and constructing unbiased decision counterfactuals. JEL Codes: C10 (Econometric and statistical methods and methodology), C55 (Large datasets: Modeling and analysis), K40 (Legal procedure, the legal system, and illegal behavior).

Individual differences in normal body temperature: longitudinal big data analysis of patient records.

OBJECTIVE: To estimate individual level body temperature and to correlate it with other measures of physiology and health. DESIGN: Observational cohort study. SETTING: Outpatient clinics of a large academic hospital, 2009-14. PARTICIPANTS: 35 488 patients who neither received a diagnosis for infections nor were prescribed antibiotics, in whom temperature was expected to be within normal limits. MAIN OUTCOME MEASURES: Baseline temperatures at individual level, estimated using random effects regression and controlling for ambient conditions at the time of measurement, body site, and time factors. Baseline temperatures were correlated with demographics, medical comorbidities, vital signs, and subsequent one year mortality. RESULTS: In a diverse cohort of 35 488 patients (mean age 52.9 years, 64% women, 41% non-white race) with 243 506 temperature measurements, mean temperature was 36.6°C (95% range 35.7-37.3°C, 99% range 35.3-37.7°C). Several demographic factors were linked to individual level temperature, with older people the coolest (-0.021°C for every decade, P<0.001) and African-American women the hottest (versus white men: 0.052°C, P<0.001). Several comorbidities were linked to lower temperature (eg, hypothyroidism: -0.013°C, P=0.01) or higher temperature (eg, cancer: 0.020, P<0.001), as were physiological measurements (eg, body mass index: 0.002 per m/kg2, P<0.001). Overall, measured factors collectively explained only 8.2% of individual temperature variation. Despite this, unexplained temperature variation was a significant predictor of subsequent mortality: controlling for all measured factors, an increase of 0.149°C (1 SD of individual temperature in the data) was linked to 8.4% higher one year mortality (P=0.014). CONCLUSIONS: Individuals' baseline temperatures showed meaningful variation that was not due solely to measurement error or environmental factors. Baseline temperatures correlated with demographics, comorbid conditions, and physiology, but these factors explained only a small part of individual temperature variation. Unexplained variation in baseline temperature, however, strongly predicted mortality.