Management of infodemics in outbreaks or health crises: a systematic review.

Introduction: The World Health Organization (WHO) defined an infodemic as an overabundance of information, accurate or not, in the digital and physical space, accompanying an acute health event such as an outbreak or epidemic. It can impact people's risk perceptions, trust, and confidence in the health system, and health workers. As an immediate response, the WHO developed the infodemic management (IM) frameworks, research agenda, intervention frameworks, competencies, and processes for reference by health authorities. Objective: This systematic review explored the response to and during acute health events by health authorities and other organizations operating in health. It also assessed the effectiveness of the current interventions. Methods: On 26 June 2023, an online database search included Medline (Ovid), Embase, Cochrane Library, Scopus, Epistemonikos, and the WHO website. It included English-only, peer-reviewed studies or reports covering IM processes applied by health organizations that reported their effectiveness. There was no restriction on publication dates. Two independent reviewers conducted all screening, inclusion, and quality assessments, and a third reviewer arbitrated any disagreement between the two reviewers. Results: Reviewers identified 945 records. After a final assessment, 29 studies were included in the review and were published between 2021 and 2023. Some countries (Pakistan, Yemen, Spain, Italy, Hong Kong, Japan, South Korea, Singapore, United Kingdom, United States, New Zealand, Finland, South Korea, and Russia) applied different methods of IM to people's behaviors. These included but were not limited to launching media and TV conservations, using web and scientific database searches, posting science-based COVID-19 information, implementing online surveys, and creating an innovative ecosystem of digital tools, and an Early AI-supported response with Social Listening (EARS) platform. Most of the interventions were effective in containing the harmful effects of COVID-19 infodemic. However, the quality of the evidence was not robust. Discussion: Most of the infodemic interventions applied during COVID-19 fall within the recommended actions of the WHO IM ecosystem. As a result, the study suggests that more research is needed into the challenges facing health systems in different operational environments and country contexts in relation to designing, implementing, and evaluating IM interventions, strategies, policies, and systems.

Expressions of pandemic fatigue on digital platforms: a thematic analysis of sentiment and narratives for infodemic insights.

BACKGROUND: The infodemic accompanying the COVID-19 pandemic has led to an overwhelming amount of information, including questions, concerns and misinformation. Pandemic fatigue has been identified as a concern from early in the pandemic. With new and ongoing health emergencies in 2022, it is important to understand how pandemic fatigue is being discussed and expressed by users on digital channels. This study aims to explore and report on key narrative themes associated with expressions of pandemic fatigue by users on digital platforms. METHODS: This paper describes the collection of publicly available data over a 3-month period from multiple online sources using the Meltwater and CrowdTangle platforms to source data from Twitter, Facebook, Instagram, YouTube, TikTok, Pinterest, Product Reviews, Twitch, blogs & forums. A comprehensive search strategy was developed and tested. A total of 1,484,042 social media posts were identified during the time-period that included the defined search terms for pandemic fatigue. These data were initially sorted by highest levels of engagement and from this dataset, analysts reviewed the identified posts to isolate and remove irrelevant content and identify dominant narratives. A thematic analysis was carried out on these narratives to identify themes related to expression of pandemic fatigue. Two researchers reviewed the data and themes. RESULTS: The thematic analysis of narratives identified six main themes relating to expression of pandemic fatigue, and one theme of counter narratives against pandemic fatigue. Data volume increased concurrent with the time of the mpox emergency announcement. Emergent themes showed the different ways users expressed pandemic fatigue and how it was interlaced with issues of trust, preventative measure acceptance and uptake, misinformation, and being overwhelmed with multiple or sustained emergencies. CONCLUSIONS: This paper has identified the different ways users express pandemic fatigue on digital channels over a 3-month period. Better understanding the implications of the information environment on user's perceptions, questions, and concerns regarding pandemic and more broadly emergency fatigue is vital in identifying relevant interventions and, in the longer term, strengthening the global architecture for health emergency preparedness, prevention, readiness and resilience, as evidenced in this paper. There are clear pathways for further research, including incorporating additional languages and reviewing these themes over longer time periods.

Chatbots and COVID-19: Taking Stock of the Lessons Learned.

While digital innovation in health was already rapidly evolving, the COVID-19 pandemic has accelerated the generation of digital technology tools, such as chatbots, to help increase access to crucial health information and services to those who were cut off or had limited contact with health services. This theme issue titled "Chatbots and COVID-19" presents articles from researchers and practitioners across the globe, describing the development, implementation, and evaluation of chatbots designed to address a wide range of health concerns and services. In this editorial, we present some of the key challenges and lessons learned arising from the content of this theme issue. Most notably, we note that a stronger evidence base is needed to ensure that chatbots and other digital tools are developed to best serve the needs of population health.

Informing social media analysis for public health: a cross-sectional survey of professionals.

BACKGROUND: During the COVID-19 pandemic, the field of infodemic management has grown in response to urgent global need. Social listening is the first step in managing the infodemic, and many organizations and health systems have implemented processes. Social media analysis tools have traditionally been developed for commercial purposes, rather than public health, and little is known of the experiences and needs of those professionals using them for infodemic management. METHODS: We developed a cross sectional survey and distributed through global infodemic management networks between December 2022 and February 2023. Questions were structured over four sections related to work-practice and user needs and did not collect any personal details from participants. Descriptive analysis was conducted on the study results. Qualitative analysis was used to categorise and understand answers to open-text questions. RESULTS: There were 417 participants, 162/417 who completed all survey questions, and 255/417 who completed some, all responses are included in analysis. Respondents came from all global regions and a variety of workplaces. Participants had an average of 4.4 years' experience in the analysis of social media for public health. COVID-19 was the most common health issue people had conducted social media analysis for. Results reveal a range of training, technical capacity, and support needs. CONCLUSIONS: This paper is the first we are aware of to seek and describe the needs of those using social media analysis platforms for public health purposes since the start of the COVID-19 pandemic. There are key areas for future work and research, including addressing the training, capacity building and leadership needs of those working in this space, and the need to facilitate easier access to better platforms for performing social media analysis.

Using Machine Learning Technology (Early Artificial Intelligence-Supported Response With Social Listening Platform) to Enhance Digital Social Understanding for the COVID-19 Infodemic: Development and Implementation Study.

BACKGROUND: Amid the COVID-19 pandemic, there has been a need for rapid social understanding to inform infodemic management and response. Although social media analysis platforms have traditionally been designed for commercial brands for marketing and sales purposes, they have been underused and adapted for a comprehensive understanding of social dynamics in areas such as public health. Traditional systems have challenges for public health use, and new tools and innovative methods are required. The World Health Organization Early Artificial Intelligence-Supported Response with Social Listening (EARS) platform was developed to overcome some of these challenges. OBJECTIVE: This paper describes the development of the EARS platform, including data sourcing, development, and validation of a machine learning categorization approach, as well as the results from the pilot study. METHODS: Data for EARS are collected daily from web-based conversations in publicly available sources in 9 languages. Public health and social media experts developed a taxonomy to categorize COVID-19 narratives into 5 relevant main categories and 41 subcategories. We developed a semisupervised machine learning algorithm to categorize social media posts into categories and various filters. To validate the results obtained by the machine learning-based approach, we compared it to a search-filter approach, applying Boolean queries with the same amount of information and measured the recall and precision. Hotelling T2 was used to determine the effect of the classification method on the combined variables. RESULTS: The EARS platform was developed, validated, and applied to characterize conversations regarding COVID-19 since December 2020. A total of 215,469,045 social posts were collected for processing from December 2020 to February 2022. The machine learning algorithm outperformed the Boolean search filters method for precision and recall in both English and Spanish languages (P<.001). Demographic and other filters provided useful insights on data, and the gender split of users in the platform was largely consistent with population-level data on social media use. CONCLUSIONS: The EARS platform was developed to address the changing needs of public health analysts during the COVID-19 pandemic. The application of public health taxonomy and artificial intelligence technology to a user-friendly social listening platform, accessible directly by analysts, is a significant step in better enabling understanding of global narratives. The platform was designed for scalability; iterations and new countries and languages have been added. This research has shown that a machine learning approach is more accurate than using only keywords and has the benefit of categorizing and understanding large amounts of digital social data during an infodemic. Further technical developments are needed and planned for continuous improvements, to meet the challenges in the generation of infodemic insights from social media for infodemic managers and public health professionals.

Measuring the burden of infodemics with a research toolkit for connecting information exposure, trust, and health behaviours.

BACKGROUND: During a public health emergency, accurate and useful information can be drowned out by questions, concerns, information voids, conflicting information, and misinformation. Very few studies connect information exposure and trust to health behaviours, which limits available evidence to inform when and where to act to mitigate the burden of infodemics, especially in low resource settings. This research describes the features of a toolkit that can support studies linking information exposure to health behaviours at the individual level. METHODS: To meet the needs of the research community, we determined the functional and non-functional requirements of a research toolkit that can be used in studies measuring topic-specific information exposure and health behaviours. Most data-driven infodemiology research is designed to characterise content rather than measure associations between information exposure and health behaviours. Studies also tend to be limited to specific social media platforms, are unable to capture the breadth of individual information exposure that occur online and offline, and cannot measure differences in trust by information source or content. Studies are also designed very differently, limiting synthesis of results. RESULTS: We demonstrate a way to address these requirements via a web-based study platform that includes an app that participants use to record topic-specific information exposure, a browser plugin for tracking access to relevant webpages, questionnaires that can be delivered at any time during a study, and app-based incentives for participation such as visual analytics to compare trust levels with other participants. Other features of the platform include the ability to tailor studies to local contexts, ease of use for participants, and frictionless sharing of de-identified data for aggregating individual participant data in international meta-analyses. CONCLUSIONS: Our proposed solution will be able to capture detailed data about information exposure and health behaviour data, standardise study design while simultaneously supporting localisation, and make it easy to synthesise individual participant data across studies. Future research will need to evaluate the toolkit in realistic scenarios to understand the usability of the toolkit for both participants and investigators.

Establishing Infodemic Management in Germany: A Framework for Social Listening and Integrated Analysis to Report Infodemic Insights at the National Public Health Institute.

BACKGROUND: To respond to the need to establish infodemic management functions at the national public health institute in Germany (Robert Koch Institute, RKI), we explored and assessed available data sources, developed a social listening and integrated analysis framework, and defined when infodemic management functions should be activated during emergencies. OBJECTIVE: We aimed to establish a framework for social listening and integrated analysis for public health in the German context using international examples and technical guidance documents for infodemic management. METHODS: This study completed the following objectives: identified (potentially) available data sources for social listening and integrated analysis; assessed these data sources for their suitability and usefulness for integrated analysis in addition to an assessment of their risk using the RKI's standardized data protection requirements; developed a framework and workflow to combine social listening and integrated analysis to report back actionable infodemic insights for public health communications by the RKI and stakeholders; and defined criteria for activating integrated analysis structures in the context of a specific health event or health emergency. RESULTS: We included and classified 38% (16/42) of the identified and assessed data sources for social listening and integrated analysis at the RKI into 3 categories: social media and web-based listening data, RKI-specific data, and infodemic insights. Most data sources can be analyzed weekly to detect current trends and narratives and to inform a timely response by reporting insights that include a risk assessment and scalar judgments of different narratives and themes. CONCLUSIONS: This study identified, assessed, and prioritized a wide range of data sources for social listening and integrated analysis to report actionable infodemic insights, ensuring a valuable first step in establishing and operationalizing infodemic management at the RKI. This case study also serves as a roadmap for others. Ultimately, once operational, these activities will inform better and targeted public health communication at the RKI and beyond.

Social Media Analysis Tools for Public Health: A Cross-Sectional Survey.

During the COVID-19 pandemic the field of infodemic management has grown significantly. Social listening is the first step in managing the infodemic but little is known of the experience of public health professionals using social media analysis tools for health. Our survey sought the views of infodemic managers. Participants (n=417) had an average of 4.4 years' experience in social media analysis for health. Results reveal gaps in technical capabilities of tools, data sources, and languages covered. For future planning for infodemic preparednessand preventi on it is vital to understand and deliver for analysis needs of those working in the field.

Infodemic Insights on Trust in a Health Emergency: A Narrative Deep-Dive.

Trust in authorities is important during health emergencies, and there are many factors that influence this. The infodemic has resulted in overwhelming amounts of information being shared on digital media during the COVID-19 pandemic, and this research looked at trust-related narratives during a one-year period. We identified three key findings related to trust and distrust narratives, and a country-level comparison showed less mistrust narratives in a country with a higher level of trust in government. Trust is a complex construct and the findings of this study present results that warrant further exploration.

An Adaptive Digital Intelligence System to Support Infodemic Management: The WHO EARS Platform.

The WHO Early AI-Supported Response with Social Listening (EARS) platform was developed to help inform infodemic response during the COVID-19 pandemic. There was continual monitoring and evaluation of the platform and feedback from end-users was sought on a continual basis. Iterations were made to the platform in response to user needs, including the introduction of new languages and countries, and additional features to better enable more fine-grained and rapid analysis and reporting. The platform demonstrates how a scalable, adaptable system can be iterated upon to continue to support those working in emergency preparedness and response.

Narrative Trends over the COVID-19 Pandemic: Digital Social Listening to Inform WHO Infodemic Management.

The COVID-19 infodemic is an overwhelming amount of information that has challenged pandemic communication and epidemic response. WHO has produced weekly infodemic insights reports to identify questions, concerns, information voids expressed and experienced by people online. Publicly available data was collected and categorized to a public health taxonomy to enable thematic analysis. Analysis showed three key periods of narrative volume peaks. Understanding how conversations change over time can help inform future infodemic preparedness and prevention planning.

The Good Talk! A Serious Game to Boost People's Competence to Have Open Conversations About COVID-19: Protocol for a Randomized Controlled Trial.

BACKGROUND: Vaccine hesitancy is one of the many factors impeding efforts to control the COVID-19 pandemic. Exacerbated by the COVID-19 infodemic, misinformation has undermined public trust in vaccination, led to greater polarization, and resulted in a high social cost where close social relationships have experienced conflict or disagreements about the public health response. OBJECTIVE: The purpose of this paper is to describe the theory behind the development of a digital behavioral science intervention-The Good Talk!-designed to target vaccine-hesitant individuals through their close contacts (eg, family, friends, and colleagues) and to describe the methodology of a research study to evaluate its efficacy. METHODS: The Good Talk! uses an educational serious game approach to boost the skills and competences of vaccine advocates to have open conversations about COVID-19 with their close contacts who are vaccine hesitant. The game teaches vaccine advocates evidence-based open conversation skills to help them speak with individuals who have opposing points of view or who may ascribe to nonscientifically supported beliefs while retaining trust, identifying common ground, and fostering acceptance and respect of divergent views. The game is currently under development and will be available on the web, free to access for participants worldwide, and accompanied by a promotional campaign to recruit participants through social media channels. This protocol describes the methodology for a randomized controlled trial that will compare participants who play The Good Talk! game with a control group that plays the widely known noneducational game Tetris. The study will evaluate a participant's open conversation skills, self-efficacy, and behavioral intentions to have an open conversation with a vaccine-hesitant individual both before and after game play. RESULTS: Recruitment will commence in early 2023 and will cease once 450 participants complete the study (225 per group). The primary outcome is improvement in open conversation skills. Secondary outcomes are self-efficacy and behavioral intentions to have an open conversation with a vaccine-hesitant individual. Exploratory analyses will examine the effect of the game on implementation intentions as well as potential covariates or subgroup differences based on sociodemographic information or previous experiences with COVID-19 vaccination conversations. CONCLUSIONS: The outcome of the project is to promote more open conversations regarding COVID-19 vaccination. We hope that our approach will encourage more governments and public health experts to engage in their mission to reach their citizens directly with digital health solutions and to consider such interventions as an important tool in infodemic management. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/40753.

Empowering Health Workers to Build Public Trust in Vaccination: Experience from the International Pediatric Association's Online Vaccine Trust Course, 2020-2021.

BACKGROUND: The quality of interactions between health workers (HWs) and caregivers is key in vaccine acceptance. To optimize this, HWs need knowledge about best vaccine communication practices in person and on social media. Most pre-service curricula do not include such approaches. COVID-19 necessitated the International Pediatric Association (IPA) to shift from in-person train the trainer workshops to developing an online Vaccine Trust Course to address these gaps. METHOD: The seven-module, 8-hour Vaccine Trust Course was offered online in seven languages and promoted globally. Course outcomes for participants between September 1, 2020 and September 30, 2021 were assessed using enrollment, participation, and completion data; pre-and post-training surveys of attitudes, knowledge, and practice skills; and follow-up practice surveys 3 months post course completion. RESULTS: Of the 4,926 participants across 137 countries who registered; 2,381 (48.3 %) started the course, with 1,217 (51.1 %) completing. The majority were 25 - 39 years (57 %), female (57 %), and in pediatrics (70 %); 31 % came from India. 62 % of completers rated course structure/design as excellent, 36 % as good. Over 80 % rated the content as the most valuable aspect. Three months post training, 61 % HWs reported increased empathy towards caregivers, confidence while counseling and increased vaccine acceptance amongst their patients. 21 % identified the course as the only factor in these positive changes. CONCLUSION: Shifting from face-to-face to online training due to the COVID-19 pandemic helped increase the global reach of HWs course engagement and uptake. Trained HWs reported increased empathy towards caregivers and confidence while counseling and increased patient vaccine acceptance.

WHO competency framework for health authorities and institutions to manage infodemics: its development and features.

BACKGROUND: In April 2020, the World Health Organization (WHO) Information Network for Epidemics produced an agenda for managing the COVID-19 infodemic. "Infodemic" refers to the overabundance of information-including mis- and disinformation. In this agenda it was pointed out the need to create a competency framework for infodemic management (IM). This framework was released by WHO on 20th September 2021. This paper presents the WHO framework for IM by highlighting the different investigative steps behind its development. METHODS: The framework was built through three steps. Step 1 included the preparatory work following the guidelines in the Guide to writing Competency Framework for WHO Academy courses. Step 2 was based on a qualitative study with participants (N = 25), identified worldwide on the basis of their academic background in relevant fields of IM or of their professional experience in IM activities at the institutional level. The interviews were conducted online between December 2020 and January 2021, they were video-recorded and analyzed using thematic analysis. In Step 3, two stakeholder panels were conducted to revise the framework. RESULTS: The competency framework contains four primary domains, each of which comprised main activities, related tasks, and knowledge and skills. It identifies competencies to manage and monitor infodemics, to design, conduct and evaluate appropriate interventions, as well as to strengthen health systems. Its main purpose is to assist institutions in reinforcing their IM capacities and implementing effective IM processes and actions according to their individual contexts and resources. CONCLUSION: The competency framework is not intended to be a regulatory document nor a training curriculum. As a WHO initiative, it serves as a reference tool to be applied according to local priorities and needs within the different countries. This framework can assist institutions in strengthening IM capacity by hiring, staff development, and human resources planning.

Infodemics: A new challenge for public health.

The COVID-19 information epidemic, or "infodemic," demonstrates how unlimited access to information may confuse and influence behaviors during a health emergency. However, the study of infodemics is relatively new, and little is known about their relationship with epidemics management. Here, we discuss unresolved issues and propose research directions to enhance preparedness for future health crises.

Infodemic Signal Detection During the COVID-19 Pandemic: Development of a Methodology for Identifying Potential Information Voids in Online Conversations.

BACKGROUND: The COVID-19 pandemic has been accompanied by an infodemic: excess information, including false or misleading information, in digital and physical environments during an acute public health event. This infodemic is leading to confusion and risk-taking behaviors that can be harmful to health, as well as to mistrust in health authorities and public health responses. The World Health Organization (WHO) is working to develop tools to provide an evidence-based response to the infodemic, enabling prioritization of health response activities. OBJECTIVE: In this work, we aimed to develop a practical, structured approach to identify narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, thus providing actionable data to help the WHO prioritize its response efforts to address the COVID-19 infodemic. METHODS: We developed a taxonomy to filter global public conversations in English and French related to COVID-19 on social media into 5 categories with 35 subcategories. The taxonomy and its implementation were validated for retrieval precision and recall, and they were reviewed and adapted as language about the pandemic in online conversations changed over time. The aggregated data for each subcategory were analyzed on a weekly basis by volume, velocity, and presence of questions to detect signals of information voids with potential for confusion or where mis- or disinformation may thrive. A human analyst reviewed and identified potential information voids and sources of confusion, and quantitative data were used to provide insights on emerging narratives, influencers, and public reactions to COVID-19-related topics. RESULTS: A COVID-19 public health social listening taxonomy was developed, validated, and applied to filter relevant content for more focused analysis. A weekly analysis of public online conversations since March 23, 2020, enabled quantification of shifting interests in public health-related topics concerning the pandemic, and the analysis demonstrated recurring voids of verified health information. This approach therefore focuses on the detection of infodemic signals to generate actionable insights to rapidly inform decision-making for a more targeted and adaptive response, including risk communication. CONCLUSIONS: This approach has been successfully applied to identify and analyze infodemic signals, particularly information voids, to inform the COVID-19 pandemic response. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently recur and narratives shift over time. The approach is being piloted in individual countries and WHO regions to generate localized insights and actions; meanwhile, a pilot of an artificial intelligence-based social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Beyond the COVID-19 pandemic, the taxonomy and methodology may be adapted for fast deployment in future public health events, and they could form the basis of a routine social listening program for health preparedness and response planning.

WHO Digital Intelligence Analysis for Tracking Narratives and Information Voids in the COVID-19 Infodemic.

The COVID-19 pandemic is the first to unfold in the highly digitalized society of the 21st century and is therefore the first pandemic to benefit from and be threatened by a thriving real-time digital information ecosystem. For this reason, the response to the infodemic required development of a public health social listening taxonomy, a structure that can simplify the chaotic information ecosystem to enable an adaptable monitoring infrastructure that detects signals of fertile ground for misinformation and guides trusted sources of verified information to fill in information voids in a timely manner. A weekly analysis of public online conversations since 23 March 2020 has enabled the quantification of running shifts of public interest in public health-related topics concerning the pandemic and has demonstrated the frequent resumption of information voids relevant for public health interventions and risk communication in an emergency response setting.

EARS - A WHO Platform for AI-Supported Real-Time Online Social Listening of COVID-19 Conversations.

As the COVID-19 pandemic evolves, the accompanying infodemic is being amplified through social media and has challenged effective response. The WHO Early AI-supported Response with Social Listening (EARS) is a platform that summarizes real-time information about how people are talking about COVID-19 in public spaces online in 20 pilot countries and in four languages. The aim of the platform is to better integrate social listening with other data sources and analyses that can inform infodemic response.