Causal language and strength of inference in academic and media articles shared in social media (CLAIMS): A systematic review.

BACKGROUND: The pathway from evidence generation to consumption contains many steps which can lead to overstatement or misinformation. The proliferation of internet-based health news may encourage selection of media and academic research articles that overstate strength of causal inference. We investigated the state of causal inference in health research as it appears at the end of the pathway, at the point of social media consumption. METHODS: We screened the NewsWhip Insights database for the most shared media articles on Facebook and Twitter reporting about peer-reviewed academic studies associating an exposure with a health outcome in 2015, extracting the 50 most-shared academic articles and media articles covering them. We designed and utilized a review tool to systematically assess and summarize studies' strength of causal inference, including generalizability, potential confounders, and methods used. These were then compared with the strength of causal language used to describe results in both academic and media articles. Two randomly assigned independent reviewers and one arbitrating reviewer from a pool of 21 reviewers assessed each article. RESULTS: We accepted the most shared 64 media articles pertaining to 50 academic articles for review, representing 68% of Facebook and 45% of Twitter shares in 2015. Thirty-four percent of academic studies and 48% of media articles used language that reviewers considered too strong for their strength of causal inference. Seventy percent of academic studies were considered low or very low strength of inference, with only 6% considered high or very high strength of causal inference. The most severe issues with academic studies' causal inference were reported to be omitted confounding variables and generalizability. Fifty-eight percent of media articles were found to have inaccurately reported the question, results, intervention, or population of the academic study. CONCLUSIONS: We find a large disparity between the strength of language as presented to the research consumer and the underlying strength of causal inference among the studies most widely shared on social media. However, because this sample was designed to be representative of the articles selected and shared on social media, it is unlikely to be representative of all academic and media work. More research is needed to determine how academic institutions, media organizations, and social network sharing patterns impact causal inference and language as received by the research consumer.

Cash for carbon: A randomized trial of payments for ecosystem services to reduce deforestation.

We evaluated a program of payments for ecosystem services in Uganda that offered forest-owning households annual payments of 70,000 Ugandan shillings per hectare if they conserved their forest. The program was implemented as a randomized controlled trial in 121 villages, 60 of which received the program for 2 years. The primary outcome was the change in land area covered by trees, measured by classifying high-resolution satellite imagery. We found that tree cover declined by 4.2% during the study period in treatment villages, compared to 9.1% in control villages. We found no evidence that enrollees shifted their deforestation to nearby land. We valued the delayed carbon dioxide emissions and found that this program benefit is 2.4 times as large as the program costs.