Clinical decision support system in emergency telephone triage: A scoping review of technical design, implementation and evaluation.

OBJECTIVES: Emergency department overcrowding could be improved by upstream telephone triage. Emergency telephone triage aims at managing and orientating adequately patients as early as possible and distributing limited supply of staff and materials. This complex task could be improved with the use of Clinical decision support systems (CDSS). The aim of this scoping review was to identify literature gaps for the future development and evaluation of CDSS for Emergency telephone triage. MATERIALS AND METHODS: We present here a scoping review of CDSS designed for emergency telephone triage, and compared them in terms of functional characteristics, technical design, health care implementation and methodologies used for evaluation, following the PRISMA-ScR guidelines. RESULTS: Regarding design, 19 CDSS were retrieved: 12 were knowledge based CDSS (decisional algorithms built according to guidelines or clinical expertise) and 7 were data driven (statistical, machine learning, or deep learning models). Most of them aimed at assisting nurses or non-medical staff by providing patient orientation and/or severity/priority assessment. Eleven were implemented in real life, and only three were connected to the Electronic Health Record. Regarding evaluation, CDSS were assessed through various aspects: intrinsic characteristics, impact on clinical practice or user apprehension. Only one pragmatic trial and one randomized controlled trial were conducted. CONCLUSION: This review highlights the potential of a hybrid system, user tailored, flexible, connected to the electronic health record, which could work with oral, video and digital data; and the need to evaluate CDSS on intrinsic characteristics and impact on clinical practice, iteratively at each distinct stage of the IT lifecycle.

Model and Strategy for Predicting and Discovering Drug-Drug Interactions.

Taking several medications at the same time is an increasingly common phenomenon in our society. The combination of drugs is certainly not without risk of potentially dangerous interactions. Taking into account all possible interactions is a very complex task as it is not yet known what all possible interactions between drugs and their types are. Machine learning based models have been developed to help with this task. However, the output of these models is not structured enough to be integrated in a clinical reasoning process on interactions. In this work, we propose a clinically relevant and technically feasible model and strategy for drug interactions.

Putting undergraduate medical students in AI-CDSS designers' shoes: An innovative teaching method to develop digital health critical thinking.

INTRODUCTION: Digital health programs are urgently needed to accelerate the adoption of Artificial Intelligence and Clinical Decision Support Systems (AI-CDSS) in clinical settings. However, such programs are still lacking for undergraduate medical students, and new approaches are required to prepare them for the arrival of new and unknown technologies. At University Paris Cité medical school, we designed an innovative program to develop the digital health critical thinking of undergraduate medical students that consisted of putting medical students in AI-CDSS designers' shoes. METHODS: We followed the six steps of Kern's approach for curriculum development: identification of needs, definition of objectives, design of an educational strategy, implementation, development of an assessment and design of program evaluation. RESULTS: A stand-alone and elective AI-CDSS program was implemented for fourth-year medical students. Each session was designed from an AI-CDSS designer viewpoint, with theoretical and practical teaching and brainstorming time on a project that consisted of designing an AI-CDSS in small groups. From 2021 to 2022, 15 students were enrolled: they rated the program 4.4/5, and 80% recommended it. Seventy-four percent considered that they had acquired new skills useful for clinical practice, and 66% felt more confident with technologies. The AI-CDSS program aroused great enthusiasm and strong engagement of students: 8 designed an AI-CDSS and wrote two scientific 5-page articles presented at the Medical Informatics Europe conference; 4 students were involved in a CDSS research project; 2 students asked for a hospital internship in digital health; and 1 decided to pursue PhD training. DISCUSSION: Putting students in AI-CDSS designers' shoes seemed to be a fruitful and innovative strategy to develop digital health skills and critical thinking toward AI technologies. We expect that such programs could help future doctors work in rapidly evolving digitalized environments and position themselves as key leaders in digital health.

Determining the Set of Items to Include in Breast Operative Reports, Using Clustering Algorithms on Retrospective Data Extracted from Clinical DataWarehouse.

Medical reports are key elements to guarantee the quality, and continuity of care but their quality remains an issue. Standardization and structuration of reports can increase their quality, but are usually based on expert opinions. Here, we hypothesize that a structured model of medical reports could be learnt using machine learning on retrospective medical reports extracted from clinical data warehouses (CDW). To investigate our hypothesis, we extracted breast cancer operative reports from our CDW. Each document was preprocessed and split into sentences. Clustering was performed using TFIDF, Paraphrase or Universal Sentence Encoder along with K-Means, DBSCAN, or Hierarchical clustering. The best couple was TFIDF/K-Means, providing a sentence coverage of 89 % on our dataset; and allowing to identify 7 main categories of items to include in breast cancer operative reports. These results are encouraging for a document preset creation task and should then be validated and implemented in real life.

Translating the Observational Medical Outcomes Partnership - Common Data Model (OMOP-CDM) Electronic Health Records to an OWL Ontology.

The heterogeneity of electronic health records model is a major problem: it is necessary to gather data from various models for clinical research, but also for clinical decision support. The Observational Medical Outcomes Partnership - Common Data Model (OMOP-CDM) has emerged as a standard model for structuring health records populated from various other sources. This model is proposed as a relational database schema. However, in the field of decision support, formal ontologies are commonly used. In this paper, we propose a translation of OMOP-CDM into an ontology, and we explore the utility of the semantic web for structuring EHR in a clinical decision support perspective, and the use of the SPARQL language for querying health records. The resulting ontology is available online.

Design of an Ontology-Based Triage System for Patients with Chronic Pain.

OBJECTIVE: Waiting time for a consultation for chronic pain is a widespread health problem. This paper presents the design of an ontology use to assess patients referred to a consultation for chronic pain. METHODS: We designed OntoDol, an ontology of pain domain for patient triage based on priority degrees. Terms were extracted from clinical practice guidelines and mapped to SNOMED-CT concepts through the Python module Owlready2. Selected SNOMED-CT concepts, relationships, and the TIME ontology, were implemented in the ontology using Protégé. Decision rules were implemented with SWRL. We evaluated OntoDol on 5 virtual cases. RESULTS: OntoDol contains 762 classes, 92 object properties and 18 SWRL rules to assign patients to 4 categories of priority. OntoDol was able to assert every case and classify them in the right category of priority. CONCLUSION: Further works will extend OntoDol to other diseases and assess OntoDol with real world data from the hospital.

A Qualitative Method for Learning Medical Expert Reasoning.

The aim of this paper is to propose a qualitative method for learning a model that represents the closest possible experts reasoning and strategies to provide recommendations of antibiotics. The learned model contains an integrity constraint and a preference formula. The former indicates the features that an antibiotic should have to be recommended. The later indicates the rank of recommendation of an antibiotic.

Deep Neural Networks for Simultaneously Capturing Public Topics and Sentiments During a Pandemic: Application on a COVID-19 Tweet Data Set.

BACKGROUND: Public engagement is a key element for mitigating pandemics, and a good understanding of public opinion could help to encourage the successful adoption of public health measures by the population. In past years, deep learning has been increasingly applied to the analysis of text from social networks. However, most of the developed approaches can only capture topics or sentiments alone but not both together. OBJECTIVE: Here, we aimed to develop a new approach, based on deep neural networks, for simultaneously capturing public topics and sentiments and applied it to tweets sent just after the announcement of the COVID-19 pandemic by the World Health Organization (WHO). METHODS: A total of 1,386,496 tweets were collected, preprocessed, and split with a ratio of 80:20 into training and validation sets, respectively. We combined lexicons and convolutional neural networks to improve sentiment prediction. The trained model achieved an overall accuracy of 81% and a precision of 82% and was able to capture simultaneously the weighted words associated with a predicted sentiment intensity score. These outputs were then visualized via an interactive and customizable web interface based on a word cloud representation. Using word cloud analysis, we captured the main topics for extreme positive and negative sentiment intensity scores. RESULTS: In reaction to the announcement of the pandemic by the WHO, 6 negative and 5 positive topics were discussed on Twitter. Twitter users seemed to be worried about the international situation, economic consequences, and medical situation. Conversely, they seemed to be satisfied with the commitment of medical and social workers and with the collaboration between people. CONCLUSIONS: We propose a new method based on deep neural networks for simultaneously extracting public topics and sentiments from tweets. This method could be helpful for monitoring public opinion during crises such as pandemics.

Towards a Clinical Decision Support System for Helping Medical Students in Emergency Call Centers.

In critical situations such as pandemic, medical students are often called to help in emergency call centers. However, they may encounter difficulties in phone triage because of a lack of medical skills. Here, we aim at developing a Clinical Decision Support System for helping medical students in phone call triage of pediatric patients. The system is based on the PAT (Pediatric Assessment Triangle) and local guidelines. It is composed of two interfaces. The first allows a quick assessment of severity signs, and the second provides recommendations and additional elements such as "elements to keep in mind" or "medical advice to give to patient". The system was evaluated by 20 medical students, with two fictive clinical cases. 75% of them found the content useful and clear, and the navigation easy. 65% would feel more reassured to have this system in emergency call centers. Further works are planned to improve the system before implementation in real-life.

An Interactive Interface for Displaying Recommendations on Emergency Phone Triage in Pediatrics.

Emergency phone triage aims at identifying quickly patients with critical emergencies. Patient triage is not an easy task, especially in situations involving children, mostly due to the lack of training and the lack of clinical guidelines for children. To overcome these issues, we aim at designing and assessing an interactive interface for displaying recommendations on emergency phone triage in pediatrics. Four medical students formalized local guidelines written by the SAMU of Paris, into a decision tree and designed an interface according to usability principles. The navigation within the interface was designed to allow the identification of critical emergencies at the beginning of the decision process, and thus ensuring a quick response in case of critical emergencies. The interface was assessed by 10 medical doctors: they appreciated the ergonomics (e.g., intuitive colors), and found easy to navigate through the interface. Nine of them would like to use this interface during phone call triage. In the future, this interface will be improved and implemented in emergency call centers.

Speak-PIM, Towards a Framework for the Automatic Detection of Potentially Inappropriate Prescriptions.

Potentially inappropriate medications (PIMs) have adverse health consequences, particularly in elderly patients. Various explicit criteria have been developed to detect PIMs. However, it is difficult to apply these criteria without the help of an electronic decision support tool. Programming these tools can be very complex. Indeed, for computer scientists it is difficult to understand medical issues and for clinicians it is difficult to program in a computer programming language. In this work we present Speak-PIM, a framework for formalizing the PIM's rules. Speak-PIM is based on a very simple semantics which is suitable for the declaration of PIMs without embarking on all the complexity of description logic or computer languages. It aims to offer an efficient collaboration between the computer scientists and clinicians.

Decision-support systems for managing polypharmacy in the elderly: A scoping review.

Polypharmacy, the consuming of more than five drugs, is a public health problem. It can lead to many interactions and adverse drug reactions and is very expensive. Therapeutic guidelines for managing polypharmacy in the elderly have been issued, but are highly complex, limiting their use. Decision-support systems have therefore been developed to automate the execution of these guidelines, or to provide information about drugs adapted to the context of polypharmacy. These systems differ widely in terms of their technical design, knowledge sources and evaluation methods. We present here a scoping review of electronic systems for supporting the management, by healthcare providers, of polypharmacy in elderly patients. Most existing reviews have focused mainly on evaluation results, whereas the present review also describes the technical design of these systems and the methodologies for developing and evaluating them. A systematic bibliographic search identified 19 systems differing considerably in terms of their technical design (rule-based systems, documentary approach, mixed); outputs (textual report, alerts and/or visual approaches); and evaluations (impact on clinical practices, impact on patient outcomes, efficiency and/or user satisfaction). The evaluations performed are minimal (among all the systems identified, only one system has been evaluated according to all the criteria mentioned above) and no machine learning systems and/or conflict management systems were retrieved. This review highlights the need to develop new methodologies, combining various approaches for decision support system in polypharmacy.

Clinical Decision Support Systems for Antibiotic Prescribing: An Inventory of Current French Language Tools.

Clinical decision support systems (CDSSs) are increasingly being used by clinicians to support antibiotic decision making in infection management. However, coexisting CDSSs often target different types of physicians, infectious situations, and patient profiles. The objective of this study was to perform an up-to-date inventory of French language CDSSs currently used in community and hospital settings for antimicrobial prescribing and to describe their main characteristics. A literature search, a search among smartphone application stores, and an open discussion with antimicrobial stewardship (AMS) experts were conducted in order to identify available French language CDSSs. Any clinical decision support tool that provides a personalized recommendation based on a clinical situation and/or a patient was included. Eleven CDSSs were identified through the search strategy. Of the 11 CDSSs, only 2 had been the subject of published studies, while 9 CDSSs were identified through smartphone application stores and expert knowledge. The majority of CDSSs were available free of charge (n = 8/11, 73%). Most CDSSs were accessible via smartphone applications (n = 9/11, 82%) and online websites (n = 8/11, 73%). Recommendations for antibiotic prescribing in urinary tract infections, upper and lower respiratory tract infections, and digestive tract infections were provided by over 90% of the CDSSs. More than 90% of the CDSSs displayed recommendations for antibiotic selection, prioritization, dosage, duration, route of administration, and alternative antibiotics in case of allergy. Information about antibiotic side effects, prescription recommendations for specific patient profiles and adaptation to local epidemiology were often missing or incomplete. There is a significant but heterogeneous offer for antibiotic prescribing decision support in French language. Standardized evaluation of these systems is needed to assess their impact on antimicrobial prescribing and antimicrobial resistance.

Encouraging Behavior Changes and Preventing Cardiovascular Diseases Using the Prevent Connect Mobile Health App: Conception and Evaluation of App Quality.

BACKGROUND: Cardiovascular diseases are a major cause of death worldwide. Mobile health apps could help in preventing cardiovascular diseases by improving modifiable risk factors such as eating habits, physical activity levels, and alcohol or tobacco consumption. OBJECTIVE: The aim of this study was to design a mobile health app, Prevent Connect, and to assess its quality for (1) assessing patient behavior for 4 cardiovascular risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and (2) suggesting personalized recommendations and mobile health interventions for risky behaviors. METHODS: The knowledge base of the app is based on French national recommendations for healthy eating, physical activity, and limiting alcohol and tobacco consumption. It contains a list of patient behaviors and related personalized recommendations and digital health interventions. The interface was designed according to usability principles. Its quality was assessed by a panel of 52 users in a 5-step process: completion of the demographic form, visualization of a short presentation of the app, testing of the app, completion of the user version of the Mobile App Rating Scale (uMARS), and an open group discussion. RESULTS: This app assesses patient behaviors through specific questionnaires about 4 risk factors (unhealthy eating, sedentary lifestyle, alcohol, and tobacco consumption) and suggests personalized recommendations and digital health interventions for improving behavior. The app was deemed to be of good quality, with a mean uMARS quality score of 4 on a 5-point Likert scale. The functionality and information content of the app were particularly appreciated, with a mean uMARS score above 4. Almost all the study participants appreciated the navigation system and found the app easy to use. More than three-quarters of the study participants found the app content relevant, concise, and comprehensive. The aesthetics and the engagement of the app were also appreciated (uMARS score, 3.7). Overall, 80% (42/52) of the study participants declared that the app helped them to become aware of the importance of addressing health behavior, and 65% (34/52) said that the app helped motivate them to change lifestyle habits. CONCLUSIONS: The app assessed the risky behaviors of the patients and delivered personalized recommendations and digital health interventions for multiple risk factors. The quality of the app was considered to be good, but the impact of the app on behavior changes is yet to be demonstrated and will be assessed in further studies.

ABiMed: Towards an Innovative Clinical Decision Support System for Medication Reviews and Polypharmacy Management.

Polypharmacy in elderly is a public health problem with both clinical (increase of adverse drug events) and economic issues. One solution is medication review, a structured assessment of patients' drug orders by the pharmacist for optimizing the therapy. However, this task is tedious, cognitively complex and error-prone, and only a few clinical decision support systems have been proposed for supporting it. Existing systems are either rule-based systems implementing guidelines, or documentary systems presenting drug knowledge. In this paper, we present the ABiMed research project, and, through literature reviews and brainstorming, we identified five candidate innovations for a decision support system for medication review: patient data transfer from GP to pharmacists, use of semantic technologies, association of rule-based and documentary approaches, use of machine learning, and a two-way discussion between pharmacist and GP after the medication review.

General practitioners' perceptions of using virtual primary care during the COVID-19 pandemic: An international cross-sectional survey study.

With the onset of COVID-19, general practitioners (GPs) and patients worldwide swiftly transitioned from face-to-face to digital remote consultations. There is a need to evaluate how this global shift has impacted patient care, healthcare providers, patient and carer experience, and health systems. We explored GPs' perspectives on the main benefits and challenges of using digital virtual care. GPs across 20 countries completed an online questionnaire between June-September 2020. GPs' perceptions of main barriers and challenges were explored using free-text questions. Thematic analysis was used to analyse the data. A total of 1,605 respondents participated in our survey. The benefits identified included reducing COVID-19 transmission risks, guaranteeing access and continuity of care, improved efficiency, faster access to care, improved convenience and communication with patients, greater work flexibility for providers, and hastening the digital transformation of primary care and accompanying legal frameworks. Main challenges included patients' preference for face-to-face consultations, digital exclusion, lack of physical examinations, clinical uncertainty, delays in diagnosis and treatment, overuse and misuse of digital virtual care, and unsuitability for certain types of consultations. Other challenges include the lack of formal guidance, higher workloads, remuneration issues, organisational culture, technical difficulties, implementation and financial issues, and regulatory weaknesses. At the frontline of care delivery, GPs can provide important insights on what worked well, why, and how during the pandemic. Lessons learned can be used to inform the adoption of improved virtual care solutions and support the long-term development of platforms that are more technologically robust and secure.

A framework for validating AI in precision medicine: considerations from the European ITFoC consortium.

BACKGROUND: Artificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks. METHODS: The European "ITFoC (Information Technology for the Future Of Cancer)" consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology. RESULTS: This framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the "ITFoC Challenge". This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets. CONCLUSIONS: The predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.

Evaluating the Impact of COVID-19 on the Adoption of Virtual Care in General Practice in 20 Countries (inSIGHT): Protocol and Rationale Study.

BACKGROUND: In recent decades, virtual care has emerged as a promising option to support primary care delivery. However, despite the potential, adoption rates remained low. With the outbreak of COVID-19, it has suddenly been pushed to the forefront of care delivery. As we progress into the second year of the COVID-19 pandemic, there is a need and opportunity to review the impact remote care had in primary care settings and reassess its potential future role. OBJECTIVE: This study aims to explore the perspectives of general practitioners (GPs) and family doctors on the (1) use of virtual care during the COVID-19 pandemic, (2) perceived impact on quality and safety of care, and (3) essential factors for high-quality and sustainable use of virtual care in the future. METHODS: This study used an online cross-sectional questionnaire completed by GPs distributed across 20 countries. The survey was hosted in Qualtrics and distributed using email, social media, and the researchers' personal contact networks. GPs were eligible for the survey if they were working mainly in primary care during the period of the COVID-19 pandemic. Descriptive statistical analysis will be performed for quantitative variables, and relationships between the use of virtual care and perceptions on impact on quality and safety of care and participants' characteristics may be explored. Qualitative data (free-text responses) will be analyzed using framework analysis. RESULTS: Data collection took place from June 2020 to September 2020. As of this manuscript's submission, a total of 1605 GP respondents participated in the questionnaire. Further data analysis is currently ongoing. CONCLUSIONS: The study will provide a comprehensive overview of the availability of virtual care technologies, perceived impact on quality and safety of care, and essential factors for high-quality future use. In addition, a description of the underlying factors that influence this adoption and perceptions, in both individual GP and family doctor characteristics and the context in which they work, will be provided. While the COVID-19 pandemic may prove the first great stress test of the capabilities, capacity, and robustness of digital systems currently in use, remote care will likely remain an increasingly common approach in the future. There is an imperative to identify the main lessons from this unexpected transformation and use them to inform policy decisions and health service design. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/30099.

A Web Interface for Antibiotic Prescription Recommendations in Primary Care: User-Centered Design Approach.

BACKGROUND: Antibiotic misuse is a serious public health problem worldwide. National health authorities release clinical practice guidelines (CPGs) to guide general practitioners (GPs) in their choice of antibiotics. However, despite the large-scale dissemination of CPGs, GPs continue to prescribe antibiotics that are not recommended as first-line treatments. This nonadherence to recommendations may be due to GPs misunderstanding the CPGs. A web interface displaying antibiotic prescription recommendations and their justifications could help to improve the comprehensibility and readability of CPGs, thereby increasing the adoption of recommendations regarding antibiotic treatment. OBJECTIVE: This study aims to design and evaluate a web interface for antibiotic prescription displaying both the recommended antibiotics and their justifications in the form of antibiotic properties. METHODS: A web interface was designed according to the same principles as e-commerce interfaces and was assessed by 117 GPs. These GPs were asked to answer 17 questions relating to the usefulness, user-friendliness, and comprehensibility and readability of the interface, and their satisfaction with it. Responses were recorded on a 4-point Likert scale (ranging from "absolutely disagree" to "absolutely agree"). At the end of the evaluation, the GPs were allowed to provide optional, additional free comments. RESULTS: The antibiotic prescription web interface consists of three main sections: a clinical summary section, a filter section, and a recommended antibiotics section. The majority of GPs appreciated the clinical summary (90/117, 76.9%) and filter (98/117, 83.8%) sections, whereas 48.7% (57/117) of them reported difficulty reading some of the icons in the recommended antibiotics section. Overall, 82.9% (97/117) of GPs found the display of drug properties useful, and 65.8% (77/117) reported that the web interface improved their understanding of CPG recommendations. CONCLUSIONS: The web interface displaying antibiotic recommendations and their properties can help doctors understand the rationale underlying CPG recommendations regarding antibiotic treatment, but further improvements are required before its implementation into a clinical decision support system.

Reorganisation of GP surgeries during the COVID-19 outbreak: analysis of guidelines from 15 countries.

BACKGROUND: General practitioners (GPs) play a key role in managing the COVID-19 outbreak. However, they may encounter difficulties adapting their practices to the pandemic. We provide here an analysis of guidelines for the reorganisation of GP surgeries during the beginning of the pandemic from 15 countries. METHODS: A network of GPs collaborated together in a three-step process: (i) identification of key recommendations of GP surgery reorganisation, according to WHO, CDC and health professional resources from health care facilities; (ii) collection of key recommendations included in the guidelines published in 15 countries; (iii) analysis, comparison and synthesis of the results. RESULTS: Recommendations for the reorganisation of GP surgeries of four types were identified: (i) reorganisation of GP consultations (cancelation of non-urgent consultations, follow-up via e-consultations), (ii) reorganisation of GP surgeries (area partitioning, visual alerts and signs, strict hygiene measures), (iii) reorganisation of medical examinations by GPs (equipment, hygiene, partial clinical examinations, patient education), (iv) reorganisation of GP staff (equipment, management, meetings, collaboration with the local community). CONCLUSIONS: We provide here an analysis of guidelines for the reorganisation of GP surgeries during the beginning of the COVID-19 outbreak from 15 countries. These guidelines focus principally on clinical care, with less attention paid to staff management, and the area of epidemiological surveillance and research is largely neglected. The differences of guidelines between countries and the difficulty to apply them in routine care, highlight the need of advanced research in primary care. Thereby, primary care would be able to provide recommendations adapted to the real-world settings and with stronger evidence, which is especially necessary during pandemics.