Combining Crowd-Sourcing and Automated Content Methods to Improve Estimates of Overall Media Coverage: Theme Mentions in E-cigarette and Other Tobacco Coverage.

Exposure to media content can shape public opinions about tobacco. Accurately describing content is a first step to showing such effects. Historically, content analyses have hand-coded tobacco-focused texts from a few media sources which ignored passing mention coverage and social media sources, and could not reliably capture over-time variation. By using a combination of crowd-sourced and automated coding, we labeled the population of all e-cigarette and other tobacco-related (including cigarettes, hookah, cigars, etc.) 'long-form texts' (focused and passing coverage, in mass media and website articles) and social media items (tweets and YouTube videos) collected May 2014-June 2017 for four tobacco control themes. Automated coding of theme coverage met thresholds for item-level precision and recall, event validation, and weekly-level reliability for most sources, except YouTube. Health, Policy, Addiction and Youth themes were frequent in e-cigarette long-form focused coverage (44%-68%), but not in long-form passing coverage (5%-22%). These themes were less frequent in other tobacco coverage (long-form focused (13-32%) and passing coverage (4-11%)). Themes were infrequent in both e-cigarette (1-3%) and other tobacco tweets (2-4%). Findings demonstrate that passing e-cigarette and other tobacco long-form coverage and social media sources paint different pictures of theme coverage than focused long-form coverage. Automated coding also allowed us to code the amount of data required to estimate reliable weekly theme coverage over three years. E-cigarette theme coverage showed much more week-to-week variation than did other tobacco coverage. Automated coding allows accurate descriptions of theme coverage in passing mentions, social media, and trends in weekly theme coverage.

Validating the Hornik & Woolf approach to choosing media campaign themes: Do promising beliefs predict behavior change in a longitudinal study?

Hornik and Woolf (1999) proposed using cross-sectional survey data to prioritize beliefs to address with communication campaign messages. The empirical component of the approach combines evidence of (1) association of beliefs with intentions and (2) current level of beliefs to calculate a 'percentage to gain' as the potential promise of a belief. However, the method relies on cross-sectional data; its conclusions are open to challenge. Here, a panel study assesses whether the calculated promise of a belief actually predicts future behavior change. A nationally representative sample of 3,204 U.S. youth and young adults were interviewed twice, six months apart. Sixteen beliefs about the benefits and costs of smoking cigarettes are compared with regard to their percentage to gain (calculated from cross-sectional data) and their ability to account for subsequent cigarette use. A belief's cross-sectional percentage to gain is substantially associated with its ability to predict subsequent behavior change (r=.53, p<.05).

Identifying Potential Campaign Themes to Prevent Youth Initiation of E-Cigarettes.

Once a target audience and a health behavior of interest are selected for a potential mass media campaign, the next task is selecting beliefs about the health behavior to serve as the basis for campaign message content. For novel health behaviors, such as the use of emerging tobacco products, limited empirical research on beliefs about these behaviors exists. A multimethod approach was applied to generate potential campaign beliefs for emerging behaviors. Three methods were conducted in this investigation in order to generate a list of potential testable campaign beliefs, using youth e-cigarette use as a case study: (1) a search of published and unpublished literature including gathering measures from several national surveys (through 2016), (2) an online elicitation survey (conducted in 2016), and (3) unsupervised topic modeling of media texts (from 2014 to 2015, analyzed in 2016). Details are provided on how each method was employed to both generate and prioritize beliefs related to youth e-cigarette use into a final set of 115 beliefs across 23 belief themes. This multimethod approach can provide four utilities when thinking through a health campaign for novel health behaviors: (1) developing an exhaustive and complementary list of beliefs, (2) generating overarching themes and distilling larger themes into more nuanced beliefs, (3) identifying language most relevant to the target population, and (4) prioritizing beliefs for message pilot testing with members of the target audience. SUPPLEMENT INFORMATION: This article is part of a supplement entitled Fifth Anniversary Retrospective of "The Real Cost," the Food and Drug Administration's Historic Youth Smoking Prevention Media Campaign, which is sponsored by the U.S. Food and Drug Administration.