Capturing Emerging Experiential Knowledge for Vaccination Guidelines Through Natural Language Processing: Proof-of-Concept Study.

BACKGROUND: Experience-based knowledge and value considerations of health professionals, citizens, and patients are essential to formulate public health and clinical guidelines that are relevant and applicable to medical practice. Conventional methods for incorporating such knowledge into guideline development often involve a limited number of representatives and are considered to be time-consuming. Including experiential knowledge can be crucial during rapid guidance production in response to a pandemic but it is difficult to accomplish. OBJECTIVE: This proof-of-concept study explored the potential of artificial intelligence (AI)-based methods to capture experiential knowledge and value considerations from existing data channels to make these insights available for public health guideline development. METHODS: We developed and examined AI-based methods in relation to the COVID-19 vaccination guideline development in the Netherlands. We analyzed Dutch messages shared between December 2020 and June 2021 on social media and on 2 databases from the Dutch National Institute for Public Health and the Environment (RIVM), where experiences and questions regarding COVID-19 vaccination are reported. First, natural language processing (NLP) filtering techniques and an initial supervised machine learning model were developed to identify this type of knowledge in a large data set. Subsequently, structural topic modeling was performed to discern thematic patterns related to experiences with COVID-19 vaccination. RESULTS: NLP methods proved to be able to identify and analyze experience-based knowledge and value considerations in large data sets. They provide insights into a variety of experiential knowledge that is difficult to obtain otherwise for rapid guideline development. Some topics addressed by citizens, patients, and professionals can serve as direct feedback to recommendations in the guideline. For example, a topic pointed out that although travel was not considered as a reason warranting prioritization for vaccination in the national vaccination campaign, there was a considerable need for vaccines for indispensable travel, such as cross-border informal caregiving, work or study, or accessing specialized care abroad. Another example is the ambiguity regarding the definition of medical risk groups prioritized for vaccination, with many citizens not meeting the formal priority criteria while being equally at risk. Such experiential knowledge may help the early identification of problems with the guideline's application and point to frequently occurring exceptions that might initiate a revision of the guideline text. CONCLUSIONS: This proof-of-concept study presents NLP methods as viable tools to access and use experience-based knowledge and value considerations, possibly contributing to robust, equitable, and applicable guidelines. They offer a way for guideline developers to gain insights into health professionals, citizens, and patients' experience-based knowledge, especially when conventional methods are difficult to implement. AI-based methods can thus broaden the evidence and knowledge base available for rapid guideline development and may therefore be considered as an important addition to the toolbox of pandemic preparedness.

Using structural topic modelling to reveal patterns in reports on opioid drugs in a pharmacovigilance database.

BACKGROUND: Adverse drug reaction (ADR) reports in pharmacovigilance databases often contain coded information and large amounts of unstructured or semi-structured information in plain text format. The unstructured format and sheer volume of these data often render them neglected. Structural topic modelling (STM) represents a potentially insightful way of harnessing these valuable data and to detect grouping or themes in spontaneous reports to aid signal detection. PURPOSE: This was an explorative study of the potential for structural topic modelling to identify useful patterns in ADR reports involving opioid drugs in a pharmacovigilance database. METHODS: A dataset of ADR reports on opioid drugs reported to the Netherlands Pharmacovigilance Centre Lareb from 1991 to December 2020 was used, comprising a total of 3069 unique reports. Qualitative text analysis was combined with STM, an automated text analysis method, to examine these data. RESULTS: In reports submitted directly by patients and healthcare professionals, 11 meaningful topics were identified, whereby patient experience reports, particularly in relation to pain (relief), and the timing of intake and ADRs of tramadol and paracetamol, were the most common. Of the 12 topics identified in reports received via marketing authorization holders, patch and skin-related side effects, addiction and constipation were the most prevalent. CONCLUSIONS: The STM-based analysis identified information that cannot always be captured by coding with the Medical Dictionary for Regulatory Activities (MedDRA®). The identified topics reflect findings in the literature on opioids.

Kaleidoscopic integration: Advancing the integration of incommensurable knowledge in healthcare guidelines.

The integration of different types of knowledge in epistemically hierarchical settings remains one of the greatest challenges when developing standards for healthcare practices. In this article, we open up the notion of knowledge integration and empirically examine the various ways in which different types of knowledge interact and can be integrated. To allow us to focus on the diverse forms of knowledge as well as their interaction and integration, we combine Moreira's work on repertoires of evaluation with that of Dewulf and Bouwen on frame interactions. We examine the quest for knowledge integration by studying interactions in the case of the development of the COVID-19 vaccination guideline in the Netherlands, a prime example of the encounter of a wide range and diversity of knowledge that needs to be appraised and integrated into guideline recommendations. Drawing on ethnographic observations of more than 70 guideline development meetings between 2021 and 2022, we first map the different types of knowledge and reasonings used by the guideline developers and subsequently analyze their interactions. We identified eight knowledge interaction patterns, being disconnection, polarization, accommodation, incorporation, reconnection, reconciliation, passive juxtaposition, and kaleidoscopic integration. We hereby draw attention to the various possible knowledge interactions encompassed in the concept of "knowledge integration", especially to those in which integration is achieved while differences and incompatibilities are maintained. Finally, we discuss potential ways to facilitate fruitful knowledge interactions during collaborative work which include the ability to accept and sustain tensions between different types of knowledge and making more explicit use of frame or rather repertoire reflection.