Large Language Model-based Chatbot as a Source of Advice on First Aid in Heart Attack.

The ability of the cutting-edge large language model-powered chatbots to generate human-like answers to user questions hypothetically could be utilized for providing real-time advice on first aid for witnesses of cardiovascular emergencies. This study aimed to evaluate quality of the chatbot responses to inquiries on help in heart attack. The study simulated interrogation of the new Bing chatbot (Microsoft Corporation, USA) with the "heart attack what to do" prompt coming from 3 countries, the Gambia, India and the USA. The chatbot responses (20 per country) were evaluated for congruence with the International First Aid, Resuscitation, and Education Guidelines 2020 using a checklist. For all user inquiries, the chatbot provided answers containing some guidance on first aid. However, the responses commonly left out some potentially life-saving instructions, for instance to encourage the person to stop physical activity, to take antianginal medication, or to start cardiopulmonary resuscitation for unresponsive abnormally breathing person. Mean percentage of the responses having full congruence with the checklist criteria varied from 7.3 for India to 16.8 for the USA. A quarter of responses for the Gambia and the USA, and 45.0% for India contained superfluous guidelines-inconsistent directives. The chatbot advice on help in heart attack has omissions, inaccuracies and misleading instructions, and therefore the chatbot cannot be recommended as a credible source of information on first aid. Active research and organizational efforts are needed to mitigate the risk of uncontrolled misinformation and establish measures for guaranteeing trustworthiness of the chatbot-mediated counseling.

Large Language Model (LLM)-Powered Chatbots Fail to Generate Guideline-Consistent Content on Resuscitation and May Provide Potentially Harmful Advice.

INTRODUCTION: Innovative large language model (LLM)-powered chatbots, which are extremely popular nowadays, represent potential sources of information on resuscitation for the general public. For instance, the chatbot-generated advice could be used for purposes of community resuscitation education or for just-in-time informational support of untrained lay rescuers in a real-life emergency. STUDY OBJECTIVE: This study focused on assessing performance of two prominent LLM-based chatbots, particularly in terms of quality of the chatbot-generated advice on how to give help to a non-breathing victim. METHODS: In May 2023, the new Bing (Microsoft Corporation, USA) and Bard (Google LLC, USA) chatbots were inquired (n = 20 each): "What to do if someone is not breathing?" Content of the chatbots' responses was evaluated for compliance with the 2021 Resuscitation Council United Kingdom guidelines using a pre-developed checklist. RESULTS: Both chatbots provided context-dependent textual responses to the query. However, coverage of the guideline-consistent instructions on help to a non-breathing victim within the responses was poor: mean percentage of the responses completely satisfying the checklist criteria was 9.5% for Bing and 11.4% for Bard (P >.05). Essential elements of the bystander action, including early start and uninterrupted performance of chest compressions with adequate depth, rate, and chest recoil, as well as request for and use of an automated external defibrillator (AED), were missing as a rule. Moreover, 55.0% of Bard's responses contained plausible sounding, but nonsensical guidance, called artificial hallucinations, that create risk for inadequate care and harm to a victim. CONCLUSION: The LLM-powered chatbots' advice on help to a non-breathing victim omits essential details of resuscitation technique and occasionally contains deceptive, potentially harmful directives. Further research and regulatory measures are required to mitigate risks related to the chatbot-generated misinformation of public on resuscitation.

An expert consensus­based checklist for quality appraisal of educational resources on adult basic life support: a Delphi study

Objective: Given the lack of a unified tool for appraising the quality of educational resources for lay-rescuer delivery of adult basic life support (BLS), this study aimed to develop an appropriate evaluation checklist based on a consensus of international experts. Methods: In a two-round Delphi study, participating experts completed questionnaires to rate each item of a predeveloped 72-item checklist indicating agreement that an item should be utilized to evaluate the conformance of an adult BLS educational resource with resuscitation guidelines. Consensus on item inclusion was defined as a rating of ≥7 points from ≥75% of experts. Experts were encouraged to add anonymous suggestions for modifying or adding new items. Results: Of the 46 participants, 42 (91.3%) completed the first round (representatives of 25 countries with a median of 16 years of professional experience in resuscitation) and 40 (87.0%) completed the second round. Thirteen of 72 baseline items were excluded, 55 were included unchanged, four were included after modification, and four new items were added. The final checklist comprises 63 items under the subsections "safety" (one item), "recognition" (nine items), "call for help" (four items), "chest compressions" (12 items), "rescue breathing" (12 items), "defibrillation" (nine items), "continuation of CPR" (two items), "choking" (10 items) and "miscellaneous" (four items). Conclusions: The produced checklist is a ready-to-use expert consensus­based tool for appraising the quality of educational content on lay-rescuer provision of adult BLS. The checklist gives content developers a tool to ensure educational resources comply with current resuscitation knowledge, and may serve as a component of a prospective standardized international framework for quality assurance in resuscitation education.

Dr. Google's Advice on First Aid: Evaluation of the Search Engine's Question-Answering System Responses to Queries Seeking Help in Health Emergencies.

INTRODUCTION: The ever-growing penetration of internet and mobile technologies into society suggests that people will increasingly use web searches to seek health-related information, including advice on first aid in medical emergencies. When a bystander is incompetent in first aid and has no immediate support from Emergency Medical Services (EMS), as it happens in low-resource settings or in disasters, instructions found online could be the sole driver for administering first aid before arrival of professional help. STUDY OBJECTIVE: The aim of this study was to evaluate quality of advice on first aid generated by a web search engine's question-answering system (QAS) in response to search queries concerning provision of help in common health emergencies. METHODS: In December 2022-January 2023, an English-language search was carried out in Google with ten queries based on the keyword combinations (what to do OR how to help) AND (bleeding OR chest pain OR choking OR not breathing OR seizure). The search engine's QAS responses (up to 11 per search query) were evaluated for compliance with the International Federation of Red Cross First Aid Guidelines 2020 using the pre-developed checklists. RESULTS: Out of 98 QAS items generated by Google, 67.3% (n = 66) were excluded, mainly because the QAS answers did not address original queries. Eligible unique QAS responses (n = 27) showed poor coverage of the guideline-compliant instructions on first aid. Mean percentage of QAS responses providing a first aid instruction with complete adherence to the guidelines varied from 0.0 for choking to 19.5 for seizure. Only three (11.1%) QAS responses contained an explicit instruction to access EMS, while 66.7% (n = 18) included directions either contradictory to the guidelines and potentially harmful (eg, use of home remedies in chest pain) or inapplicable for an untrained person (eg, use of tourniquet in bleeding). CONCLUSION: Although the search engine's QAS responds to user's inquiries concerning assistance in health emergencies, the QAS-generated answers, as a rule, omit potentially life-saving evidence-based instructions on first aid and oftentimes give advices noncompliant with current guidelines or inadequate for untrained people, and thus create risks for causing harm to a victim.

Relationship of Public Interest in Cardiopulmonary Resuscitation with Cardiac Arrest Epidemiology and National Socioeconomic Indicators: Exploratory Infodemiology Study.

STUDY OBJECTIVE: Web-based big data analytics provides a great opportunity to measure public interest in cardiac arrest (CA) and cardiopulmonary resuscitation (CPR). This study aimed to examine associations of online interest in CPR and CA with epidemiological characteristics of out-of-hospital CA (OHCA) and national socioeconomic indicators in a set of European countries. METHODS: Country-level online search popularity data for CPR and CA topics measured in relative search volume (RSV) with Google Trends (GT), published OHCA epidemiological indicators, and World Bank's socioeconomic statistics of 28 European countries for the year 2017 were analyzed for correlation using Spearman's rank correlation coefficient (r S ). RESULTS: Whereas OHCA incidence, bystander CPR rate, and hospital survival did not correlate with RSV for CPR or CA, the rate of return of spontaneous circulation (ROSC) demonstrated a positive correlation with RSV for CPR (r S = 0.388; P = .042). Further, RSV for CPR positively correlated with countries' gross domestic product and health expenditure (r S = 0.939 and 0.566; P ≤.002) and negatively correlated with mortality caused by road traffic injury (r S = -0.412; P = .029). CONCLUSION: For the sample of European countries, public interest in CPR or CA showed no relationship with real bystander CPR rates and therefore could not be recommended as a proxy of community readiness to attempt resuscitation. The association of RSV for CPR with the rate of ROSC and countries' socioeconomic characteristics suggests it could be used for identifying geographies with poor performance of prehospital systems in terms of managing CA, in particular where effective epidemiological surveillance for CA may be unavailable.