Persuasion with Precision: Using Natural Language Processing to Improve Instrument Fidelity for Risk Communication Experimental Treatments.

Instrument fidelity in message testing research hinges upon how precisely messages operationalize treatment conditions. However, numerous message-testing studies have unmitigated threats to validity and reliability because no established procedures exist to guide construction of message treatments. Their construction typically occurs in a black box, resulting in suspect inferential conclusions about treatment effects. Because a mixed methods approach is needed to enhance instrument fidelity in message testing research, this article contributes to the field of mixed methods research by presenting an integrated multistage procedure for constructing precise message treatments using an exploratory sequential mixed methods design. This work harnesses the power of integration through crossover analysis to improve instrument fidelity in message testing research through the use of natural language processing.

Characters matter: How narratives shape affective responses to risk communication.

INTRODUCTION: Whereas scientists depend on the language of probability to relay information about hazards, risk communication may be more effective when embedding scientific information in narratives. The persuasive power of narratives is theorized to reside, in part, in narrative transportation. PURPOSE: This study seeks to advance the science of stories in risk communication by measuring real-time affective responses as a proxy indicator for narrative transportation during science messages that present scientific information in the context of narrative. METHODS: This study employed a within-subjects design in which participants (n = 90) were exposed to eight science messages regarding flood risk. Conventional science messages using probability and certainty language represented two conditions. The remaining six conditions were narrative science messages that embedded the two conventional science messages within three story forms that manipulated the narrative mechanism of character selection. Informed by the Narrative Policy Framework, the characters portrayed in the narrative science messages were hero, victim, and victim-to-hero. Natural language processing techniques were applied to identify and rank hero and victim vocabularies from 45 resident interviews conducted in the study area; the resulting classified vocabulary was used to build each of the three story types. Affective response data were collected over 12 group sessions across three flood-prone communities in Montana. Dial response technology was used to capture continuous, second-by-second recording of participants' affective responses while listening to each of the eight science messages. Message order was randomized across sessions. ANOVA and three linear mixed-effects models were estimated to test our predictions. RESULTS: First, both probabilistic and certainty science language evoked negative affective responses with no statistical differences between them. Second, narrative science messages were associated with greater variance in affective responses than conventional science messages. Third, when characters are in action, variation in the narrative mechanism of character selection leads to significantly different affective responses. Hero and victim-to-hero characters elicit positive affective responses, while victim characters produce a slightly negative response. CONCLUSIONS: In risk communication, characters matter in audience experience of narrative transportation as measured by affective responses.

Cultural Theory of Risk as a Heuristic for Understanding Perceptions of Oil and Gas Development in Eastern Montana, USA.

This paper applies Douglas' cultural theory of risk to understand perceptions of risk associated with oil and gas development in eastern Montana. Based on the analysis of interviews with 36 rural residents, findings show the dominant perception of risk is most closely aligned with an Individualist worldview. Despite direct experience with oil or wastewater spills, most interviewees described spills as "no big deal", viewed nature as resilient, and felt that the economic benefits outweigh negative impacts. Cultural theory was a useful heuristic for understanding this dominant worldview, as well as identifying points of deviation. For example, interviewees discussed the benefits of landowner associations - a more Egalitarian approach to dealing with oil companies. Some landowners relied on external authorities (e.g., sheriff) when dealing with oil companies, revealing a Hierarchical approach to issues they face. Interviewees expressed frustration with the lack of enforcement of existing regulations, which can be interpreted as either support for - or indictment of - Hierarchical solutions. While the Individualist worldview is dominant, our qualitative analysis reveals the complex tensions at work among rural residents. The results suggest areas where policymakers, advocacy groups, and residents may find common ground to address potential environmental and health risks.