Unraveling complex causal processes that affect sustainability requires more integration between empirical and modeling approaches.

Scientists seek to understand the causal processes that generate sustainability problems and determine effective solutions. Yet, causal inquiry in nature-society systems is hampered by conceptual and methodological challenges that arise from nature-society interdependencies and the complex dynamics they create. Here, we demonstrate how sustainability scientists can address these challenges and make more robust causal claims through better integration between empirical analyses and process- or agent-based modeling. To illustrate how these different epistemological traditions can be integrated, we present four studies of air pollution regulation, natural resource management, and the spread of COVID-19. The studies show how integration can improve empirical estimates of causal effects, inform future research designs and data collection, enhance understanding of the complex dynamics that underlie observed temporal patterns, and elucidate causal mechanisms and the contexts in which they operate. These advances in causal understanding can help sustainability scientists develop better theories of phenomena where social and ecological processes are dynamically intertwined and prior causal knowledge and data are limited. The improved causal understanding also enhances governance by helping scientists and practitioners choose among potential interventions, decide when and how the timing of an intervention matters, and anticipate unexpected outcomes. Methodological integration, however, requires skills and efforts of all involved to learn how members of the respective other tradition think and analyze nature-society systems.

Community-based monitoring to facilitate water management by local institutions in Costa Rica.

Water scarcity is a global problem that can be compounded by inefficient water management, including underinvestment in infrastructure, underpricing of water use, and underenforcement of user rules. Here, we explore whether these inefficiencies can be reduced in rural Costa Rica via an externally driven community monitoring program (i.e., a program initiated by an outside organization and run by citizens). The monitoring program aimed to reduce groundwater extraction from aquifers, as well as to improve water quality and user satisfaction, by supplying additional information about field conditions and additional scrutiny of user and management authority activities and by fostering citizen engagement in water management. Using a specially designed smartphone application (app) and WhatsApp, monitors could report weekly on the conditions of the water system, including service disruptions, water quality, leaks, and source contamination. The app automatically compiled the individual reports into a summary report, which was then made available to the community water management committees and water users. The program was randomly implemented in 80 of 161 communities that expressed an interest in participating. One year after the program started, we detect modest, albeit imprecisely estimated, effects of the program in the predicted directions: less groundwater extracted, better water quality, and more satisfied users. Although the estimated effects are imprecise, the monitoring program appears to be equally or more cost effective for reducing groundwater extraction than another program in the same region that encouraged households to adopt water-efficient technologies.

Adoption of community monitoring improves common pool resource management across contexts.

Pervasive overuse and degradation of common pool resources (CPRs) is a global concern. To sustainably manage CPRs, effective governance institutions are essential. A large literature has developed to describe the institutional design features employed by communities that successfully manage their CPRs. Yet, these designs remain far from universally adopted. We focus on one prominent institutional design feature, community monitoring, and ask whether nongovernmental organizations or governments can facilitate its adoption and whether adoption of monitoring affects CPR use. To answer these questions, we implemented randomized controlled trials in six countries. The harmonized trials randomly assigned the introduction of community monitoring to 400 communities, with data collection in an additional 347 control communities. Most of the 400 communities adopted regular monitoring practices over the course of a year. In a meta-analysis of the experimental results from the six sites, we find that the community monitoring reduced CPR use and increased user satisfaction and knowledge by modest amounts. Our findings demonstrate that community monitoring can improve CPR management in disparate contexts, even when monitoring is externally initiated rather than homegrown. These findings provide guidance for the design of future programs and policies intended to develop monitoring capabilities in communities. Furthermore, our harmonized, multisite trial provides sustainability science with a new way to study the complexity of socioecological systems and builds generalizable insights about how to improve CPR management.

The choice of message and messenger to drive behavior change that averts the health impacts of wildfires: an online randomized controlled experiment.

BACKGROUND: To reduce the negative health effects from wildfire smoke exposure, effective risk and health communication strategies are vital. We estimated the behavioral effects from changes in message framing and messenger in public health messages about wildfire smoke on Facebook. METHODS: During September and October 2021, we conducted a preregistered online randomized controlled experiment in Facebook. Adult Facebook users (n = 1,838,100), living in nine wildfire-prone Western U.S. states, were randomly assigned to see one of two ad versions (narrative frame vs. informational frame) from one of two messengers (government vs. academic). We estimated the effects of narrative framing, the messenger, and their interactions on ad click-through rates, a measure of recipient information-seeking behavior. RESULTS: Narrative frame increased click-through rates by 25.3% (95% CI = 22.2, 28.4%), with larger estimated effects among males, recipients in areas with less frequent exposure to heavy wildfire smoke, and in areas where predominant political party affiliation of registered voters was Republican (although not statistically different from predominantly-Democrat areas). The estimated effect from an academic messenger compared to a government messenger was small and statistically nonsignificant (2.2%; 95% CI = - 0.3, 4.7%). The estimated interaction effect between the narrative framing and the academic messenger was also small and statistically nonsignificant (3.9%; 95% CI = - 1.1, 9.1%). CONCLUSIONS: Traditional public service announcements rely heavily on communicating facts (informational framing). Shifting from a fact-focused, informational framing to a story-focused, narrative framing could lead to more effective health communication in areas at risk of wildfires and in public health contexts more broadly. TRIAL REGISTRATION: Date registered: August 19, 2021; Registration DOI: https://doi.org/10.17605/OSF.IO/JMWUF.

Causal inference in coupled human and natural systems.

Coupled human and natural systems (CHANS) are complex, dynamic, interconnected systems with feedback across social and environmental dimensions. This feedback leads to formidable challenges for causal inference. Two significant challenges involve assumptions about excludability and the absence of interference. These two assumptions have been largely unexplored in the CHANS literature, but when either is violated, causal inferences from observable data are difficult to interpret. To explore their plausibility, structural knowledge of the system is requisite, as is an explicit recognition that most causal variables in CHANS affect a coupled pairing of environmental and human elements. In a large CHANS literature that evaluates marine protected areas, nearly 200 studies attempt to make causal claims, but few address the excludability assumption. To examine the relevance of interference in CHANS, we develop a stylized simulation of a marine CHANS with shocks that can represent policy interventions, ecological disturbances, and technological disasters. Human and capital mobility in CHANS is both a cause of interference, which biases inferences about causal effects, and a moderator of the causal effects themselves. No perfect solutions exist for satisfying excludability and interference assumptions in CHANS. To elucidate causal relationships in CHANS, multiple approaches will be needed for a given causal question, with the aim of identifying sources of bias in each approach and then triangulating on credible inferences. Within CHANS research, and sustainability science more generally, the path to accumulating an evidence base on causal relationships requires skills and knowledge from many disciplines and effective academic-practitioner collaborations.

Estimating the impacts of conservation on ecosystem services and poverty by integrating modeling and evaluation.

Scholars have made great advances in modeling and mapping ecosystem services, and in assigning economic values to these services. This modeling and valuation scholarship is often disconnected from evidence about how actual conservation programs have affected ecosystem services, however. Without a stronger evidence base, decision makers find it difficult to use the insights from modeling and valuation to design effective policies and programs. To strengthen the evidence base, scholars have advanced our understanding of the causal pathways between conservation actions and environmental outcomes, but their studies measure impacts on imperfect proxies for ecosystem services (e.g., avoidance of deforestation). To be useful to decision makers, these impacts must be translated into changes in ecosystem services and values. To illustrate how this translation can be done, we estimated the impacts of protected areas in Brazil, Costa Rica, Indonesia, and Thailand on carbon storage in forests. We found that protected areas in these conservation hotspots have stored at least an additional 1,000 Mt of CO2 in forests and have delivered ecosystem services worth at least $5 billion. This aggregate impact masks important spatial heterogeneity, however. Moreover, the spatial variability of impacts on carbon storage is the not the same as the spatial variability of impacts on avoided deforestation. These findings lead us to describe a research program that extends our framework to study other ecosystem services, to uncover the mechanisms by which ecosystem protection benefits humans, and to tie cost-benefit analyses to conservation planning so that we can obtain the greatest return on scarce conservation funds.

Quantifying causal mechanisms to determine how protected areas affect poverty through changes in ecosystem services and infrastructure.

To develop effective environmental policies, we must understand the mechanisms through which the policies affect social and environmental outcomes. Unfortunately, empirical evidence about these mechanisms is limited, and little guidance for quantifying them exists. We develop an approach to quantifying the mechanisms through which protected areas affect poverty. We focus on three mechanisms: changes in tourism and recreational services; changes in infrastructure in the form of road networks, health clinics, and schools; and changes in regulating and provisioning ecosystem services and foregone production activities that arise from land-use restrictions. The contributions of ecotourism and other ecosystem services to poverty alleviation in the context of a real environmental program have not yet been empirically estimated. Nearly two-thirds of the poverty reduction associated with the establishment of Costa Rican protected areas is causally attributable to opportunities afforded by tourism. Although protected areas reduced deforestation and increased regrowth, these land cover changes neither reduced nor exacerbated poverty, on average. Protected areas did not, on average, affect our measures of infrastructure and thus did not contribute to poverty reduction through this mechanism. We attribute the remaining poverty reduction to unobserved dimensions of our mechanisms or to other mechanisms. Our study empirically estimates previously unidentified contributions of ecotourism and other ecosystem services to poverty alleviation in the context of a real environmental program. We demonstrate that, with existing data and appropriate empirical methods, conservation scientists and policymakers can begin to elucidate the mechanisms through which ecosystem conservation programs affect human welfare.

Using counterfactuals to evaluate the cost-effectiveness of controlling biological invasions.

Prioritizing limited conservation funds for controlling biological invasions requires accurate estimates of the effectiveness of interventions to remove invasive species and their cost-effectiveness (cost per unit area or individual). Despite billions of dollars spent controlling biological invasions worldwide, it is unclear whether those efforts are effective, and cost-effective. The paucity of evidence results from the difficulty in measuring the effect of invasive species removal: a researcher must estimate the difference in outcomes (e.g. invasive species cover) between where the removal program intervened and what might have been observed if the program had not intervened. In the program evaluation literature, this is called a counterfactual analysis, which formally compares what actually happened and what would have happened in the absence of an intervention. When program implementation is not randomized, estimating counterfactual outcomes is especially difficult. We show how a thorough understanding of program implementation, combined with a matching empirical design can improve the way counterfactual outcomes are estimated in nonexperimental contexts. As a practical demonstration, we estimated the cost-effectiveness of South Africa's Working for Water program, arguably the world's most ambitious invasive species control program, in removing invasive alien trees from different land use types, across a large area in the Cape Floristic Region. We estimated that the proportion of the treatment area covered by invasive trees would have been 49% higher (5.5% instead of 2.7% of the grid cells occupied) had the program not intervened. Our estimates of cost per hectare to remove invasive species, however, are three to five times higher than the predictions made when the program was initiated. Had there been no control (counter-factual), invasive trees would have spread on untransformed land, but not on land parcels containing plantations or land transformed by agriculture or human settlements. This implies that the program might have prevented a larger area from being invaded if it had focused all of its clearing effort on untransformed land. Our results show that, with appropriate empirical designs, it is possible to better evaluate the impacts of invasive species removal and therefore to learn from past experiences.

Improving credibility and transparency of conservation impact evaluations through the partial identification approach.

The fundamental challenge of evaluating the impact of conservation interventions is that researchers must estimate the difference between the outcome after an intervention occurred and what the outcome would have been without it (counterfactual). Because the counterfactual is unobservable, researchers must make an untestable assumption that some units (e.g., organisms or sites) that were not exposed to the intervention can be used as a surrogate for the counterfactual (control). The conventional approach is to make a point estimate (i.e., single number along with a confidence interval) of impact, using, for example, regression. Point estimates provide powerful conclusions, but in nonexperimental contexts they depend on strong assumptions about the counterfactual that often lack transparency and credibility. An alternative approach, called partial identification (PI), is to first estimate what the counterfactual bounds would be if the weakest possible assumptions were made. Then, one narrows the bounds by using stronger but credible assumptions based on an understanding of why units were selected for the intervention and how they might respond to it. We applied this approach and compared it with conventional approaches by estimating the impact of a conservation program that removed invasive trees in part of the Cape Floristic Region. Even when we used our largest PI impact estimate, the program's control costs were 1.4 times higher than previously estimated. PI holds promise for applications in conservation science because it encourages researchers to better understand and account for treatment selection biases; can offer insights into the plausibility of conventional point-estimate approaches; could reduce the problem of advocacy in science; might be easier for stakeholders to agree on a bounded estimate than a point estimate where impacts are contentious; and requires only basic arithmetic skills.

Conditions associated with protected area success in conservation and poverty reduction.

Protected areas are the dominant approach to protecting biodiversity and the supply of ecosystem services. Because these protected areas are often placed in regions with widespread poverty and because they can limit agricultural development and exploitation of natural resources, concerns have been raised about their potential to create or reinforce poverty traps. Previous studies suggest that the protected area systems in Costa Rica and Thailand, on average, reduced deforestation and alleviated poverty. We examine these results in more detail by characterizing the heterogeneity of responses to protection conditional on observable characteristics. We find no evidence that protected areas trap historically poorer areas in poverty. In fact, we find that poorer areas at baseline seem to have the greatest levels of poverty reduction as a result of protection. However, we do find that the spatial characteristics associated with the most poverty alleviation are not necessarily the characteristics associated with the most avoided deforestation. We show how an understanding of these spatially heterogeneous responses to protection can be used to generate suitability maps that identify locations in which both environmental and poverty alleviation goals are most likely to be achieved.

Protected areas reduced poverty in Costa Rica and Thailand.

As global efforts to protect ecosystems expand, the socioeconomic impact of protected areas on neighboring human communities continues to be a source of intense debate. The debate persists because previous studies do not directly measure socioeconomic outcomes and do not use appropriate comparison groups to account for potential confounders. We illustrate an approach using comprehensive national datasets and quasi-experimental matching methods. We estimate impacts of protected area systems on poverty in Costa Rica and Thailand and find that although communities near protected areas are indeed substantially poorer than national averages, an analysis based on comparison with appropriate controls does not support the hypothesis that these differences can be attributed to protected areas. In contrast, the results indicate that the net impact of ecosystem protection was to alleviate poverty.

Empirical evidence of widespread exaggeration bias and selective reporting in ecology.

In many scientific disciplines, common research practices have led to unreliable and exaggerated evidence about scientific phenomena. Here we describe some of these practices and quantify their pervasiveness in recent ecology publications in five popular journals. In an analysis of over 350 studies published between 2018 and 2020, we detect empirical evidence of exaggeration bias and selective reporting of statistically significant results. This evidence implies that the published effect sizes in ecology journals exaggerate the importance of the ecological relationships that they aim to quantify. An exaggerated evidence base hinders the ability of empirical ecology to reliably contribute to science, policy, and management. To increase the credibility of ecology research, we describe a set of actions that ecologists should take, including changes to scientific norms about what high-quality ecology looks like and expectations about what high-quality studies can deliver.

Create a culture of experiments in environmental programs.

Organizations need a better "learning by doing" approach.

Clarifying the effect of biodiversity on productivity in natural ecosystems with longitudinal data and methods for causal inference.

Causal effects of biodiversity on ecosystem functions can be estimated using experimental or observational designs - designs that pose a tradeoff between drawing credible causal inferences from correlations and drawing generalizable inferences. Here, we develop a design that reduces this tradeoff and revisits the question of how plant species diversity affects productivity. Our design leverages longitudinal data from 43 grasslands in 11 countries and approaches borrowed from fields outside of ecology to draw causal inferences from observational data. Contrary to many prior studies, we estimate that increases in plot-level species richness caused productivity to decline: a 10% increase in richness decreased productivity by 2.4%, 95% CI [-4.1, -0.74]. This contradiction stems from two sources. First, prior observational studies incompletely control for confounding factors. Second, most experiments plant fewer rare and non-native species than exist in nature. Although increases in native, dominant species increased productivity, increases in rare and non-native species decreased productivity, making the average effect negative in our study. By reducing the tradeoff between experimental and observational designs, our study demonstrates how observational studies can complement prior ecological experiments and inform future ones.

Private costs of carbon emissions abatement by limiting beef consumption and vehicle use in the United States.

A popular strategy for mitigating climate change is to persuade or incentivize individuals to limit behaviors associated with high greenhouse gas emissions. In this study, adults in the mid-Atlantic United States bid in an auction to receive compensation for eliminating beef consumption or limiting vehicle use. The auction incentivized participants to reveal their true costs of accepting these limits for periods ranging from one week to one year. Compliance with the conditions of the auction was confirmed via a random field audit of the behavioral changes. The estimated median abatement costs were greater than $600 per tCO2e for beef consumption and $1,300 per tCO2e for vehicle use, values much higher than the price of carbon offsets and most estimates of the social cost of carbon. Although these values may decline over time with experience or broader social adoption, they imply that policies that encourage innovations to reduce the costs of behavior change, such as meat alternatives or emission-free vehicles, may be a more fruitful than those that limit beef consumption or vehicle use.

Measuring the effectiveness of protected area networks in reducing deforestation.

Global efforts to reduce tropical deforestation rely heavily on the establishment of protected areas. Measuring the effectiveness of these areas is difficult because the amount of deforestation that would have occurred in the absence of legal protection cannot be directly observed. Conventional methods of evaluating the effectiveness of protected areas can be biased because protection is not randomly assigned and because protection can induce deforestation spillovers (displacement) to neighboring forests. We demonstrate that estimates of effectiveness can be substantially improved by controlling for biases along dimensions that are observable, measuring spatial spillovers, and testing the sensitivity of estimates to potential hidden biases. We apply matching methods to evaluate the impact on deforestation of Costa Rica's renowned protected-area system between 1960 and 1997. We find that protection reduced deforestation: approximately 10% of the protected forests would have been deforested had they not been protected. Conventional approaches to evaluating conservation impact, which fail to control for observable covariates correlated with both protection and deforestation, substantially overestimate avoided deforestation (by over 65%, based on our estimates). We also find that deforestation spillovers from protected to unprotected forests are negligible. Our conclusions are robust to potential hidden bias, as well as to changes in modeling assumptions. Our results show that, with appropriate empirical methods, conservation scientists and policy makers can better understand the relationships between human and natural systems and can use this to guide their attempts to protect critical ecosystem services.

Causal assumptions and causal inference in ecological experiments.

Causal inferences from experimental data are often justified based on treatment randomization. However, inferring causality from data also requires complementary causal assumptions, which have been formalized by scholars of causality but not widely discussed in ecology. While ecologists have recognized challenges to inferring causal relationships in experiments and developed solutions, they lack a general framework to identify and address them. We review four assumptions required to infer causality from experiments and provide design-based and statistically based solutions for when these assumptions are violated. We conclude that there is no clear demarcation between experimental and non-experimental designs. This insight can help ecologists design better experiments and remove barriers between experimental and observational scholarship in ecology.

Conservation outreach that acknowledges human contributions to climate change does not inhibit action by U.S. farmers: Evidence from a large randomized controlled trial embedded in a federal program on soil health.

Technologies and practices that reduce the environmental impacts of US agriculture are well documented. Less is known about how best to encourage their adoption. We report on the results of a large randomized controlled trial conducted with nearly 10,000 agricultural producers in the United States. The experiment was embedded in US Department of Agriculture outreach efforts to improve soil conservation practices. USDA varied the content of mailings to test two sets of competing theories about outreach to agricultural producers. Contrary to conventional wisdom, we find no evidence that acknowledging the link between climate change and agricultural production discourages conservation action. Furthermore, we find that producers who were invited to a webinar were less likely to take any action to learn more about conservation practices than producers who were not told about the webinar, a result that runs counter to the popular wisdom that offering more options leads to more action.