Development and Evaluation of an Online Toolkit for Managers of 9-1-1 Emergency Communications Centers to Reduce Occupational Stress.

This article describes the development and evaluation of an online workplace stress reduction toolkit for use by managers of 9-1-1 emergency communication centers (ECCs). A three-step process for development and testing of digital learning resources was used: (1) establishing need and focus through ECC manager stakeholder engagement, (2) pretesting of the toolkit with the target ECC manager audience, and (3) toolkit utilization and evaluation. The toolkit was developed in close partnership with stakeholders throughout the entire process. Toolkit usage was documented via registration data. The evaluation utilized an online survey that included closed and open-ended questions, which were analyzed using descriptive statistics and qualitative thematic analysis. Over a 20-month period, 274 people registered for the toolkit and, of those, 184 (67%) accessed the content. Respondents to the evaluation survey (N = 156) scored the toolkit highly on satisfaction, self-efficacy, and perceived utility measures. Survey respondents reported intent to apply toolkit content through the following: providing organizational resources to help workers take better care of themselves (41%); creating a lower stress worksite environment (35%) and sharing resources with staff to (1) reduce stress (19%), (2) support conflict resolution (21%), and (3) prevent and/or stop bullying (17%). In delivering actionable content to ECC managers, the toolkit shows promise in addressing and mitigating occupational stress in ECCs. Further research needs to determine the relationship of this strategy for reducing ECC stress.

Mixed-Methods Evaluation of the Washington State Practice Transformation Support Hub.

PURPOSE: Practice transformation initiatives have the potential to promote collaborations between public health, primary care, and behavioral health, but limited empirical evidence is available on how these programs affect participating clinical practices. OBJECTIVE: To report the findings from a mixed-methods program evaluation of the Washington Practice Transformation Support Hub (Hub), a publicly funded, multicomponent practice transformation initiative in Washington State. DESIGN: We used quantitative and qualitative methods to evaluate the impact of Hub activities on participating primary care and behavioral health practices. Pre- and posttest survey data were combined with administrative program data to understand the effect of program components. Qualitative interviews contextualized findings. SETTING: Urban and rural primary care and behavioral health practices in Washington State. PARTICIPANTS: One hundred seventy-five practices that were recruited to receive Hub coaching and facilitation from 8 coaches; of these, 13 practices and all coaches participated in key informant interviews. INTERVENTION: Practice coaching and facilitation supported by an online resource portal, from January 2017 through January 2019. MAIN OUTCOME MEASURES: Self-reported progress in specific activities in 3 practice-level domains: bidirectional integration of physical and behavioral health care (care integration); alignment with community-based services for whole-person care (clinical-community linkages); and value-based payment. RESULTS: Participation in Hub activities was associated with improvements in care integration and clinical-community linkages but not with progress toward value-based payment. Qualitative results indicated that practice progress was influenced by communication with practices, the culture of the practice, resource constraints (particularly in rural areas), and perceptions about sustainability. CONCLUSIONS: This statewide practice transformation initiative was successful in strengthening primary care and behavioral health integration and clinical-community linkages among participating practices but not value-based payment. Future practice transformation efforts may benefit from addressing barriers posed by communication, limited application of value-based payment, culture change, competing priorities, and resource limitations, particularly for rural communities.

Prescription drug monitoring program use and utility by Washington State pharmacists: A mixed-methods study.

OBJECTIVES: To explore factors and situations that influence pharmacists to use the prescription drug monitoring program (PDMP) and to characterize actions taken by pharmacists after alarming scenarios from a PDMP query. DESIGN: Explanatory sequential 2-phase mixed-methods design: (1) cross-sectional Web-based survey of Washington State pharmacists followed by (2) interviews with purposefully selected respondents to explore statistically significant quantitative findings. SETTING AND PARTICIPANTS: The study was conducted in Washington State from September 2018 to February 2019. A total of 967 Washington State pharmacists from various practice settings, including inpatient and outpatient pharmacies, participated. Ten outpatient pharmacists were interviewed in the second phase. OUTCOME MEASURES: The pharmacists reported the frequency of PDMP use, opinion on the usefulness of PDMP, and action(s) taken after a concerning PDMP report. RESULTS: The usable response rate for pharmacists with a PDMP account was 17.6% (818/4659), and usable response rate for all pharmacists was 10.4% (967/9263). PDMP use varied by race, practice setting, and employer policy on PDMP use. Among the 818 PDMP users, 396 (48%) used the database at least once during a shift. Frequent PDMP users were more likely to recommend naloxone compared with less frequent users (adjusted odds ratio 1.70 [95% CI 1.09-2.65], P = 0.02). The following 3 interview themes were identified: time, company policy, and red flags. CONCLUSION: PDMP has value to pharmacists of all practice settings studied. Frequent PDMP use may facilitate more pharmacist interventions, such as a naloxone prescription.

Destress 9-1-1-an online mindfulness-based intervention in reducing stress among emergency medical dispatchers: a randomised controlled trial.

OBJECTIVES: Emergency medical dispatchers (EMDs) experience significant stress in the workplace. Yet, interventions aimed at reducing work-related stress are difficult to implement due to the logistic challenges associated with the relatively unique EMD work environment. This investigation tested the efficacy of a 7-week online mindfulness-based intervention (MBI) tailored to the EMD workforce. METHODS: Active-duty EMDs from the USA and Canada (n=323) were randomly assigned to an intervention or wait list control condition. Participants completed surveys of stress and mindfulness at baseline, post intervention, and 3 months follow-up. Repeated measures mixed effects models were used to assess changes in stress and mindfulness. RESULTS: Differences between the intervention group and control group in pre-post changes in stress using the Calgary Symptoms of Stress Inventory were statistically significant, with a difference of -10.0 (95% CI: -14.9, -5.2, p<0.001) for change from baseline to post intervention, and a difference of -6.5 (95% CI: -11.9, -1.1, p=0.02) for change from baseline to 3 months follow-up. Change in mindfulness scores did not differ between groups. However, increases in mindfulness scores were correlated with greater reductions in stress for all participants, regardless of group (r=-0.53, p<0.001). CONCLUSIONS: Development of tailored online MBIs for employees working in challenging work environments offer a promising direction for prevention and intervention. This study found that a short, weekly online MBI for EMDs resulted in reductions in reports of stress. Implications of online MBIs in other emergency responding populations and directions for future research are discussed.

Enhancing Access to Quality Online Training to Strengthen Public Health Preparedness and Response.

OBJECTIVES: To improve access to quality online training materials developed from 2010 to 2015 by 14 Preparedness and Emergency Response Learning Centers (PERLCs) by creating quality standards and enhancing searchability through a new Web-based public health training catalog. METHODS: The PERLC-developed training materials (n = 530) were evaluated for their capability to support development of preparedness competencies as established by 2 evidence-based competency frameworks. Inclusion/exclusion criteria and evaluation guidelines regarding training quality (design, technology, and instructional components) were systematically applied to PERLC products to create a training catalog. Twenty emergency preparedness professionals pilot tested content and provided feedback to improve catalog design and function. RESULTS: Seventy-eight percent of PERLC resources (n = 413) met our quality standards for inclusion in the catalog's searchable database: 358 self-paced courses, 55 informational briefs, and other materials. Twenty-one training bundles were curated. DISCUSSION: We established quality guidelines, identified strengths and weaknesses in PERLC resources, and improved accessibility to trainings. Guidelines established by this work can be generalized to trainings outside the preparedness domain. Enhancing access to quality training resources can serve as a valuable tool for increasing emergency preparedness competence.

Impact of new technologies on stress, attrition and well-being in emergency call centers: the NextGeneration 9-1-1 study protocol.

BACKGROUND: Our public health emergency response system relies on the "first of the first responders"-the emergency call center workforce that handles the emergency needs of a public in distress. Call centers across the United States have been preparing for the "Next Generation 9-1-1" initiative, which will allow citizens to place 9-1-1 calls using a variety of digital technologies. The impacts of this initiative on a workforce that is already highly stressed is unknown. There is concern that these technology changes will increase stress, reduce job performance, contribute to maladaptive coping strategies, lower employee retention, or change morale in the workplace. Understanding these impacts to inform approaches for mitigating the health and performance risks associated with new technologies is crucial for ensuring the 911 system fulfills its mission of providing optimal emergency response to the public. METHODS: Our project is an observational, prospective cohort study framed by the first new technology that will be implemented: text-to-911 calling. Emergency center call takers will be recruited nationwide. Data will be collected by online surveys distributed at each center before text-to-911 implementation; within the first month of implementation; and 6 months after implementation. Primary outcome measures are stress as measured by the Calgary Symptoms of Stress Index, use of sick leave, job performance, and job satisfaction. Primary analyses will use mixed effects regression models and mixed effects logistic regression models to estimate the change in outcome variables associated with text-to-911 implementation. Multiple secondary analyses will examine effects of stress on absenteeism; associations between technology attitudes and stress; effects of implementation on attitudes towards technology; and mitigating effects of job demands, job satisfaction, attitudes towards workplace technology and workplace support on change in stress. DISCUSSION: Our public health dependence on this workforce for our security and safety makes it imperative that the impact of technological changes such as text-to-911 are researched so appropriate intervention efforts to can be developed. Failing to protect our 9-1-1 call takers from predictable health risks would be similar to knowingly exposing field emergency responders to a toxic situation without following OSHA required training and practice standards assuring their protection.

Protocol: a multi-level intervention program to reduce stress in 9-1-1 telecommunicators.

BACKGROUND: Nationwide, emergency response systems depend on 9-1-1 telecommunicators to prioritize, triage, and dispatch assistance to those in distress. 9-1-1 call center telecommunicators (TCs) are challenged by acute and chronic workplace stressors: tense interactions with citizen callers in crisis; overtime; shift-work; ever-changing technologies; and negative work culture, including co-worker conflict. This workforce is also subject to routine exposures to secondary traumatization while handling calls involving emergency situations and while making time urgent, high stake decisions over the phone. Our study aims to test the effectiveness of a multi-part intervention to reduce stress in 9-1-1 TCs through an online mindfulness training and a toolkit containing workplace stressor reduction resources. METHODS/DESIGN: The study employs a randomized controlled trial design with three data collection points. The multi-part intervention includes an individual-level online mindfulness training and a call center-level organizational stress reduction toolkit. 160 TCs will be recruited from 9-1-1 call centers, complete a baseline survey at enrollment, and are randomly assigned to an intervention or a control group. Intervention group participants will start a 7-week online mindfulness training developed in-house and tailored to 9-1-1 TCs and their call center environment; control participants will be "waitlisted" and start the training after the study period ends. Following the intervention group's completion of the mindfulness training, all participants complete a second survey. Next, the online toolkit with call-center wide stress reduction resources is made available to managers of all participating call centers. After 3 months, a third survey will be completed by all participants. The primary outcome is 9-1-1 TCs' self-reported symptoms of stress at three time points as measured by the C-SOSI (Calgary Symptoms of Stress Inventory). Secondary outcomes will include: perceptions of social work environment (measured by metrics of social support and network conflict); mindfulness; and perceptions of social work environment and mindfulness as mediators of stress reduction. DISCUSSION: This study will evaluate the effectiveness of an online mindfulness training and call center-wide stress reduction toolkit in reducing self-reported stress in 9-1-1 TCs. The results of this study will add to the growing body of research on worksite stress reduction programs. TRIAL REGISTRATION: ClinicalTrials.gov Registration Number: NCT02961621 Registered on November 7, 2016 (retrospectively registered).

The effects of state rules on opioid prescribing in Indiana.

BACKGROUND: Prescription opioids have been linked to over half of the 28,000 opioid overdose deaths in 2014. High rates of prescription opioid non-medical use have continued despite nearly all states implementing large-scale prescription drug monitoring programs (PDMP), which points to the need to examine the impact of state PDMP's on curbing inappropriate opioid prescribing. In the short-term, PDMPs have been associated with short-term prescribing declines. Yet little is known about how such policies differentially impact patient subgroups or are interpreted by prescribing providers. Our objective was to compare volumes of prescribed opioids before and after Indiana implemented opioid prescribing emergency rules and stratify the changes in opioid prescribing by patient and provider subgroups. METHODS: An interrupted time series analysis was conducted using data obtained from the Indiana PDMP. Prescription level data was merged with census data to characterize patient socioeconomic status. Analyses were stratified by patients' gender, age, opioid dosage, and payer. The primary outcome indicator was the total morphine equivalent dose (MED) of dispensed opioids per day in the state of Indiana. Also considered were number of unique patients, unique providers, and prescriptions; MED per transaction and per day; and number of days supplied. RESULTS: After controlling for time trends, we found that total MED for opioids decreased after implementing the new emergency rules, differing by patient gender, age, and payer. The effect was larger for males than females and almost 10 times larger for 0-20 year olds as compared to the 60+ age range. Medicare and Medicaid patients experienced more decline in prescribing than patients with private insurance. Patients with prescriptions paid for by workers' comp experienced the most significant decline. The emergency rules were associated with decline in both the number of prescribers and the number of day supply. CONCLUSIONS: Although the Indiana opioid prescribing emergency rules impacted statewide prescribing behavior across all individual patient and provider characteristics, the emergency rules' effect was not consistent across patient characteristics. Further studies are needed to assess how individual patient characteristics influence the interpretation and application of state policies on opioid prescribing.

Improving Consumer Understanding of Medical Text: Development and Validation of a New SubSimplify Algorithm to Automatically Generate Term Explanations in English and Spanish.

BACKGROUND: While health literacy is important for people to maintain good health and manage diseases, medical educational texts are often written beyond the reading level of the average individual. To mitigate this disconnect, text simplification research provides methods to increase readability and, therefore, comprehension. One method of text simplification is to isolate particularly difficult terms within a document and replace them with easier synonyms (lexical simplification) or an explanation in plain language (semantic simplification). Unfortunately, existing dictionaries are seldom complete, and consequently, resources for many difficult terms are unavailable. This is the case for English and Spanish resources. OBJECTIVE: Our objective was to automatically generate explanations for difficult terms in both English and Spanish when they are not covered by existing resources. The system we present combines existing resources for explanation generation using a novel algorithm (SubSimplify) to create additional explanations. METHODS: SubSimplify uses word-level parsing techniques and specialized medical affix dictionaries to identify the morphological units of a term and then source their definitions. While the underlying resources are different, SubSimplify applies the same principles in both languages. To evaluate our approach, we used term familiarity to identify difficult terms in English and Spanish and then generated explanations for them. For each language, we extracted 400 difficult terms from two different article types (General and Medical topics) balanced for frequency. For English terms, we compared SubSimplify's explanation with the explanations from the Consumer Health Vocabulary, WordNet Synonyms and Summaries, as well as Word Embedding Vector (WEV) synonyms. For Spanish terms, we compared the explanation to WordNet Summaries and WEV Embedding synonyms. We evaluated quality, coverage, and usefulness for the simplification provided for each term. Quality is the average score from two subject experts on a 1-4 Likert scale (two per language) for the synonyms or explanations provided by the source. Coverage is the number of terms for which a source could provide an explanation. Usefulness is the same expert score, however, with a 0 assigned when no explanations or synonyms were available for a term. RESULTS: SubSimplify resulted in quality scores of 1.64 for English (P<.001) and 1.49 for Spanish (P<.001), which were lower than those of existing resources (Consumer Health Vocabulary [CHV]=2.81). However, in coverage, SubSimplify outperforms all existing written resources, increasing the coverage from 53.0% to 80.5% in English and from 20.8% to 90.8% in Spanish (P<.001). This result means that the usefulness score of SubSimplify (1.32; P<.001) is greater than that of most existing resources (eg, CHV=0.169). CONCLUSIONS: Our approach is intended as an additional resource to existing, manually created resources. It greatly increases the number of difficult terms for which an easier alternative can be made available, resulting in greater actual usefulness.

Notifiable condition reporting practices: implications for public health agency participation in a health information exchange.

BACKGROUND: The future of notifiable condition reporting in the United States is undergoing a transformation with the increasing development of Health Information Exchanges which support electronic data-sharing and -transfer networks and the wider adoption of electronic laboratory reporting. Communicable disease report forms originating in clinics are an important source of surveillance data for public health agencies. However, problems of poor data quality and delayed submission of reports to public health agencies are common. In addition, studies of barriers and facilitators to reporting have assumed that the primary reporter is the treating physician, although the extent to which a provider is involved in the reporting workflow is unclear. We sought to better understand the barriers to and burden of notifiable condition reporting from the perspectives of the three primary groups involved in reporting workflow: providers, clinic staff who bear the principal responsibility for reporting, and the public health workers who receive and process reports from clinics. In addition, we sought to situate these findings within the context of the future of notifiable disease reporting and the potential impacts of electronic lab and medical records on the surveillance system. METHODS: Seven ambulatory care clinics and 3 public health agencies that are part of a Health Information Exchange in the state of Indiana, USA, participated in the study. Data were obtained from a survey of clinic physicians (N = 29), interviews with clinic reporters (N = 11), and interviews with public health workers (N = 9). Survey data were summarized descriptively and interview transcripts underwent qualitative analysis. RESULTS: In both clinics and public health agencies, the laboratory report initiates reporting workflow. Provider involvement with reporting primarily revolves around ordering medications to treat a condition confirmed by the lab result. In clinics, reporting is typically the responsibility of clinic reporters who vary in frequency of reporting. We found an association between frequency of reporting, reporting knowledge and perceptions of reporting burden. In both clinics and public health agencies, interruptions and delays in reporting workflow are encountered due to inaccurate or missing information and impact reporting timeliness, data quality and report completeness. Both providers and clinic reporters lack clarity regarding how data submitted by their reports are used by public health agencies. It is possible that the value of reporting may be diminished when those responsible do not perceive receiving benefit in return. This may account for the low awareness of or recollection of public health communications with clinics that we observed. Despite the high likelihood that public health advisories and guidance are based, in part, on data submitted by clinics, a direct concordance may not be recognized. CONCLUSIONS: Unlike most studies of notifiable condition reporting, this study included the clinic reporters who bear primary responsibility for completing and submitting reports to public health agencies. A primary barrier to this reporting is timely and easy access to data. It is possible that expanded adoption of electronic health record and laboratory reporting systems will improve access to this data and reduce reporting the burden. However, a complete reliance on automatic electronic extraction of data requires caution and necessitates continued interfacing with clinic reporters for the foreseeable future-particularly for notifiable conditions that are high-impact, uncommon, prone to false positive readings by labs, or are hard to verify. An important finding of this study is the association between frequency of reporting, reporting knowledge and perceptions of reporting burden. Increased automation could result in even lower reporting knowledge and familiarity with reporting requirements which could actually increase reporters' perception of notifiable condition reporting as burdensome. Another finding was of uncertainty regarding how data sent to public health agencies is used or provides clinical benefit. A strong recommendation generated by these findings is that, given their central role in reporting, clinic reporters are a significant target audience for public health outreach and education that aims to alleviate perceived reporting burden and improve reporting knowledge. In particular, communicating the benefits of public health's use of the data may reduce a perceived lack of information reciprocity between clinical and public health organizations.

Completeness and timeliness of notifiable disease reporting: a comparison of laboratory and provider reports submitted to a large county health department.

BACKGROUND: Most public health agencies expect reporting of diseases to be initiated by hospital, laboratory or clinic staff even though so-called passive approaches are known to be burdensome for reporters and produce incomplete as well as delayed reports, which can hinder assessment of disease and delay recognition of outbreaks. In this study, we analyze patterns of reporting as well as data completeness and timeliness for traditional, passive reporting of notifiable disease by two distinct sources of information: hospital and clinic staff versus clinical laboratory staff. Reports were submitted via fax machine as well as electronic health information exchange interfaces. METHODS: Data were extracted from all submitted notifiable disease reports for seven representative diseases. Reporting rates are the proportion of known cases having a corresponding case report from a provider, a faxed laboratory report or an electronic laboratory report. Reporting rates were stratified by disease and compared using McNemar's test. For key data fields on the reports, completeness was calculated as the proportion of non-blank fields. Timeliness was measured as the difference between date of laboratory confirmed diagnosis and the date the report was received by the health department. Differences in completeness and timeliness by data source were evaluated using a generalized linear model with Pearson's goodness of fit statistic. RESULTS: We assessed 13,269 reports representing 9034 unique cases. Reporting rates varied by disease with overall rates of 19.1% for providers and 84.4% for laboratories (p < 0.001). All but three of 15 data fields in provider reports were more often complete than those fields within laboratory reports (p <0.001). Laboratory reports, whether faxed or electronically sent, were received, on average, 2.2 days after diagnosis versus a week for provider reports (p <0.001). CONCLUSIONS: Despite growth in the use of electronic methods to enhance notifiable disease reporting, there still exists much room for improvement.

Exploring bi-directional and SMS messaging for communications between Public Health Agencies and their stakeholders: a qualitative study.

BACKGROUND: Communication technologies that enable bi-directional/two-way communications and cell phone texting (SMS) between public health agencies and their stakeholders may improve public health surveillance, ensure targeted distribution of alerts to hard-to-reach populations, reduce mortality and morbidity in an emergency, and enable a crucial feedback loop between public health agencies and the communities they serve. Building on prior work regarding health care provider preferences for receiving one-way public health communications by email, fax or SMS, we conducted a formative, exploratory study to understand how a bi-directional system and the incorporation of SMS in that system might be used as a strategy to send and receive messages between public health agencies and community-based organizations which serve vulnerable populations, health care providers, and public health workers. Our research question: Under what conditions and/or situations might public health agencies utilize bi-directional and/or SMS messaging for disseminating time-sensitive public health information (alerts, advisories, updates, etc.) to their stakeholders? METHODS: A mixed methods (qualitative and quantitative) study was conducted between April and July 2014. Data collection included a survey distributed to health care providers and semi-structured interviews with providers, community- and government-based organization leaders and directors, and public health agency internal workforce staff. Survey respondents and interviewees were asked about their exposure to public health messages, how these messages are received and how the information in these messages are handled, and in what situations (for example, a local vs. a national event, a pandemic or emergency vs. a health update) a bi-directional and/or SMS messaging system might improve communications between public health agencies and their stakeholder group. Interview and survey data were qualitatively analyzed. Thematic codes were quantitized into dichotomous variables of 0 or 1 on a per respondent basis to enumerate the presence or absence of each thematic code, enable quantitative analysis, and inform interpretation of findings. RESULTS: Five major themes emerged from synthesizing survey and interview results: 1) Regardless of situational context (emergency vs. non-urgent) and message recipient (stakeholder group), e-mail is a favored modality for receiving public health messages; 2) The decision to use bi-directional, SMS or multiple communication strategies is complex and public health agencies' need to manage messaging concerns/barriers and benefits for all parties; 3) Both public health agencies and their stakeholders share similar values/uses and concerns regarding two-way public health messaging and SMS; 4) Public health is highly trusted, thus thoughtful, effective messaging will ensure continuation of this goodwill; and 5) Information reciprocity between public health agencies and stakeholders who share their information is essential. CONCLUSIONS: Multiple communication strategies might be utilized but the choice of a specific strategy needs to balance message content (emergency vs. routine communications), delivery (one- vs. two-way), channel (SMS, email, etc.), and public health agency burden with stakeholder preferences and technical capabilities, all while mitigating the risk of message overload and disregard of important communications by recipients.

Use of Digital COVID-19 Exposure Notifications at a Large Gathering: Survey Analysis of Public Health Conference Attendees.

BACKGROUND: WA Notify was Washington State's smartphone-based COVID-19 digital exposure notification (EN) tool, which was used to help limit the spread of COVID-19 between November 30, 2020, and May 11, 2023. Following the 2022 Washington State Public Health Association Annual Conference, attendees who had WA Notify activated began receiving ENs alerting them to a possible COVID-19 exposure during the conference. A survey was emailed to all conference attendees to measure WA Notify adoption, mechanisms through which attendees received ENs, and self-reported engagement in protective behaviors postexposure. OBJECTIVE: This study aimed to learn more about the experiences of WA Notify adopters and nonadopters who may have been exposed to COVID-19 at a large group gathering. METHODS: A web-based survey administered through REDCap (Research Electronic Data Capture; Vanderbilt University) was sent to all attendees of the Washington State Public Health Association conference. Self-reported demographic information and characteristics of respondents were summarized. Regression models were used to estimate relative risks to compare WA Notify adoption and testing behaviors between groups. RESULTS: Of the 464 total registered attendees who were sent the survey, 205 (44%) responses were received; 201 eligible attendees were included in this analysis. Of those, 149 (74%) respondents reported having WA Notify activated on their phones at the time of the conference. Among respondents with WA Notify activated, 54% (n=77) reported learning of their potential exposure from a WA Notify EN. Respondents who reported that they did not have WA Notify activated and learned of their potential exposure via the event-wide email from conference organizers were 39% less likely to test for COVID-19 compared to respondents with WA Notify activated who learned of their potential exposure from the email (relative risk 0.61, 95% CI 0.40-0.93; P=.02), and this gap was even larger when compared to respondents who learned of their exposure from a WA Notify EN. The most commonly cited reason for not having WA Notify activated was privacy concerns (n=17, 35%), followed by not wanting to receive ENs (n=6, 12%) and being unaware of WA Notify (n=5, 10%). CONCLUSIONS: Digital EN systems are an important tool to directly and anonymously notify close contacts of potential exposures and provide guidance on the next steps in a timely manner. Given the privacy concerns, there is still a need for increasing transparency surrounding EN technology to increase uptake by the public if this technology were to be used in the future to slow the spread of communicable diseases.

Public health communications and alert fatigue.

BACKGROUND: Health care providers play a significant role in large scale health emergency planning, detection, response, recovery and communication with the public. The effectiveness of health care providers in emergency preparedness and response roles depends, in part, on public health agencies communicating information in a way that maximizes the likelihood that the message is delivered, received, deemed credible and, when appropriate, acted on. However, during an emergency, health care providers can become inundated with alerts and advisories through numerous national, state, local and professional communication channels. We conducted an alert fatigue study as a sub-study of a larger randomized controlled trial which aimed to identify the most effective methods of communicating public health messages between public health agencies and providers. We report an analysis of the effects of public health message volume/frequency on recall of specific message content and effect of rate of message communications on health care provider alert fatigue. METHODS: Health care providers enrolled in the larger study (n=528) were randomized to receive public health messages via email, fax, short message service (SMS or cell phone text messaging) or to a control group that did not receive messages. For 12 months, study messages based on real events of public health significance were sent quarterly with follow-up telephone interviews regarding message receipt and topic recall conducted 5-10 days after the message delivery date. During a pandemic when numerous messages are sent, alert fatigue may impact ability to recall whether a specific message has been received due to the "noise" created by the higher number of messages. To determine the impact of "noise" when study messages were sent, we compared health care provider recall of the study message topic to the number of local public health messages sent to health care providers. RESULTS: We calculated the mean number of messages that each provider received from local public health during the time period around each study message and provider recall of study message content. We found that recall rates were inversely proportional to the mean number of messages received per week: Every increase of one local public health message per week resulted in a statistically significant 41.2% decrease (p < 0.01), 95% CI [0.39, .87] in the odds of recalling the content of the study message. CONCLUSIONS: To our knowledge, this is the first study to document the effects of alert fatigue on health care providers' recall of information. Our results suggest that information delivered too frequently and/or repetitively through numerous communication channels may have a negative effect on the ability of health care providers to effectively recall emergency information. Keeping health care providers and other first-line responders informed during an emergency is critical. Better coordination between organizations disseminating alerts, advisories and other messages may improve the ability of health care providers to recall public health emergency messages, potentially impacting effective response to public health emergency messages.

Measuring the impact of a health information exchange intervention on provider-based notifiable disease reporting using mixed methods: a study protocol.

BACKGROUND: Health information exchange (HIE) is the electronic sharing of data and information between clinical care and public health entities. Previous research has shown that using HIE to electronically report laboratory results to public health can improve surveillance practice, yet there has been little utilization of HIE for improving provider-based disease reporting. This article describes a study protocol that uses mixed methods to evaluate an intervention to electronically pre-populate provider-based notifiable disease case reporting forms with clinical, laboratory and patient data available through an operational HIE. The evaluation seeks to: (1) identify barriers and facilitators to implementation, adoption and utilization of the intervention; (2) measure impacts on workflow, provider awareness, and end-user satisfaction; and (3) describe the contextual factors that impact the effectiveness of the intervention within heterogeneous clinical settings and the HIE. METHODS/DESIGN: The intervention will be implemented over a staggered schedule in one of the largest and oldest HIE infrastructures in the U.S., the Indiana Network for Patient Care. Evaluation will be conducted utilizing a concurrent design mixed methods framework in which qualitative methods are embedded within the quantitative methods. Quantitative data will include reporting rates, timeliness and burden and report completeness and accuracy, analyzed using interrupted time-series and other pre-post comparisons. Qualitative data regarding pre-post provider perceptions of report completeness, accuracy, and timeliness, reporting burden, data quality, benefits, utility, adoption, utilization and impact on reporting workflow will be collected using semi-structured interviews and open-ended survey items. Data will be triangulated to find convergence or agreement by cross-validating results to produce a contextualized portrayal of the facilitators and barriers to implementation and use of the intervention. DISCUSSION: By applying mixed research methods and measuring context, facilitators and barriers, and individual, organizational and data quality factors that may impact adoption and utilization of the intervention, we will document whether and how the intervention streamlines provider-based manual reporting workflows, lowers barriers to reporting, increases data completeness, improves reporting timeliness and captures a greater portion of communicable disease burden in the community.

Engagement in Protective Behaviors by Digital Exposure Notification Users During the COVID-19 Pandemic, Washington State, January-June 2021.

OBJECTIVES: Smartphone-based digital exposure notification (EN) tools were introduced during the COVID-19 pandemic to supplement strained case investigation and contact tracing efforts. We examined the influence of an EN tool implemented in Washington State, WA Notify, on user engagement in behaviors that protect against COVID-19 transmission. METHODS: From January 25 through June 30, 2021, we administered 2 surveys to WA Notify users who received notification of a possible COVID-19 exposure. The initial survey, sent when users received a notification, focused on intent to engage in protective behaviors. The follow-up survey captured data on self-reported actual engagement in protective behaviors and contact by a public health contact tracer. RESULTS: Of 1507 WA Notify users who completed the initial survey, 40.1% (n = 604) reported intending to seek COVID-19 testing and 67.1% (n = 1011) intended to watch for COVID-19 symptoms. Of 407 respondents to the follow-up survey, 57.5% (n = 234) reported getting tested and 84.3% (n = 343) reported watching for COVID-19 symptoms. Approximately 84% (n = 1266) of respondents to the initial survey received a notification from WA Notify before being reached by public health contact tracers; on follow-up, 42.5% (n = 173) of respondents reported never being contacted by public health. CONCLUSIONS: Our findings suggest that WA Notify users may initiate protective behaviors earlier than nonusers who will not know of an exposure until notified by public health or by a known contact. Digital EN tools may be a valuable addition to existing public health outbreak investigation and response activities.

Equity and Accessibility of Washington State's COVID-19 Digital Exposure Notification Tool (WA Notify): Survey and Listening Sessions Among Community Leaders.

BACKGROUND: In November 2020, WA Notify, Washington State's COVID-19 digital exposure notification tool, was launched statewide to mitigate ongoing COVID-19 transmission. WA Notify uses the Bluetooth proximity-triggered, Google/Apple Exposure Notification Express framework to distribute notifications to users who have added or activated this tool on their smartphones. This smartphone-based tool relies on sufficient population-level activation to be effective; however, little is known about its adoption among communities disproportionately impacted by the COVID-19 pandemic or what barriers might limit its adoption and use among diverse populations. OBJECTIVE: We sought to (1) conduct a formative exploration of equity-related issues that may influence the access, adoption, and use of WA Notify, as perceived by community leaders of populations disproportionately impacted by the COVID-19 pandemic; and (2) generate recommendations for promoting the equitable access to and impact of this novel intervention for these communities. METHODS: We used a 2-step data collection process to gather the perspectives of community leaders across Washington regarding the launch and implementation of WA Notify in their communities. A web-based, brief, and informational survey measured the perceptions of the community-level familiarity and effectiveness of WA Notify at slowing the spread of COVID-19 and identified potential barriers and concerns to accessing and adopting WA Notify (n=17). Semistructured listening sessions were conducted to expand upon survey findings and explore the community-level awareness, barriers, facilitators, and concerns related to activating WA Notify in greater depth (n=13). RESULTS: Our findings overlap considerably with those from previous mobile health equity studies. Digital literacy, trust, information accessibility, and misinformation were highlighted as key determinants of the adoption and use of WA Notify. Although WA Notify does not track users or share data, community leaders expressed concerns about security, data sharing, and personal privacy, which were cited as outweighing the potential benefits to adoption. Both the survey and informational sessions indicated low community-level awareness of WA Notify. Community leaders recommended the following approaches to improve engagement: tailoring informational materials for low-literacy levels, providing technology navigation, describing more clearly that WA Notify can help the community, and using trusted messengers who are already engaged with the communities to communicate about WA Notify. CONCLUSIONS: As digital public health tools, such as WA Notify, emerge to address public health problems, understanding the key determinants of adoption and incorporating equity-focused recommendations into the development, implementation, and communication efforts around these tools will be instrumental to their adoption, use, and retention.

Trading-off privacy and utility: the Washington State experience assessing the performance of a public health digital exposure notification system for coronavirus disease 2019.

OBJECTIVE: Digital exposure notifications (DEN) systems were an emergency response to the coronavirus disease 2019 (COVID-19) pandemic, harnessing smartphone-based technology to enhance conventional pandemic response strategies such as contact tracing. We identify and describe performance measurement constructs relevant to the implementation of DEN tools: (1) reach (number of users enrolled in the intervention); (2) engagement (utilization of the intervention); and (3) effectiveness in preventing transmissions of COVID-19 (impact of the intervention). We also describe WA State's experience utilizing these constructs to design data-driven evaluation approaches. METHODS: We conducted an environmental scan of DEN documentation and relevant publications. Participation in multidisciplinary collaborative environments facilitated shared learning. Compilation of available data sources and their relevance to implementation and operation workflows were synthesized to develop implementation evaluation constructs. RESULTS: We identified 8 useful performance indicators within reach, engagement, and effectiveness constructs. DISCUSSION: We use implementation science to frame the evaluation of DEN tools by linking the theoretical constructs with the metrics available in the underlying disparate, deidentified, and aggregate data infrastructure. Our challenges in developing meaningful metrics include limited data science competencies in public health, validation of analytic methodologies in the complex and evolving pandemic environment, and the lack of integration with the public health infrastructure. CONCLUSION: Continued collaboration and multidisciplinary consensus activities can improve the utility of DEN tools for future public health emergencies.