Comparison of artificial intelligence large language model chatbots in answering frequently asked questions in anaesthesia.

Background: Patients are increasingly using artificial intelligence (AI) chatbots to seek answers to medical queries. Methods: Ten frequently asked questions in anaesthesia were posed to three AI chatbots: ChatGPT4 (OpenAI), Bard (Google), and Bing Chat (Microsoft). Each chatbot's answers were evaluated in a randomised, blinded order by five residency programme directors from 15 medical institutions in the USA. Three medical content quality categories (accuracy, comprehensiveness, safety) and three communication quality categories (understandability, empathy/respect, and ethics) were scored between 1 and 5 (1 representing worst, 5 representing best). Results: ChatGPT4 and Bard outperformed Bing Chat (median [inter-quartile range] scores: 4 [3-4], 4 [3-4], and 3 [2-4], respectively; P<0.001 with all metrics combined). All AI chatbots performed poorly in accuracy (score of ≥4 by 58%, 48%, and 36% of experts for ChatGPT4, Bard, and Bing Chat, respectively), comprehensiveness (score ≥4 by 42%, 30%, and 12% of experts for ChatGPT4, Bard, and Bing Chat, respectively), and safety (score ≥4 by 50%, 40%, and 28% of experts for ChatGPT4, Bard, and Bing Chat, respectively). Notably, answers from ChatGPT4, Bard, and Bing Chat differed statistically in comprehensiveness (ChatGPT4, 3 [2-4] vs Bing Chat, 2 [2-3], P<0.001; and Bard 3 [2-4] vs Bing Chat, 2 [2-3], P=0.002). All large language model chatbots performed well with no statistical difference for understandability (P=0.24), empathy (P=0.032), and ethics (P=0.465). Conclusions: In answering anaesthesia patient frequently asked questions, the chatbots perform well on communication metrics but are suboptimal for medical content metrics. Overall, ChatGPT4 and Bard were comparable to each other, both outperforming Bing Chat.

Contemporary Academic Contributions From Anesthesiologists in Adult Critical Care Medicine.

BACKGROUND: Anesthesiology has a long relationship with critical care medicine (CCM). However, US anesthesiologists are less likely to practice CCM than non-US anesthesiologists. To date, no studies have compared academic contributions in CCM between US anesthesiologists and non-US anesthesiologists. The objective of our study was to use recent trends in critical care publications as a surrogate for academic contribution among US and non-US anesthesiologists. METHODS: Research articles published between 2010 and 2015 in 3 anesthesiology journals (Anesthesiology, Anesthesia & Analgesia, and British Journal of Anaesthesia) and 3 multidisciplinary CCM journals (Critical Care Medicine, Intensive Care Medicine, and Journal of Critical Care) were reviewed. Author information, including the primary department appointment and geographic location for the first and senior author(s), and article details, including topic and publication type, were collected. Odds ratios for having a first or senior author from the United States were calculated. Anesthesiologists' contributions in individual journals were summarized, as were trends in anesthesiology CCM publications during the 6-year study period. RESULTS: A total of 3831 articles were reviewed, with 1050 (27.4%) having US authors. Eighty-two and one-half percent of CCM articles in anesthesiology journals had a US anesthesiologist as first author, and 81% had a US anesthesiologist as senior author, while fewer CCM articles in multidisciplinary journals had a US anesthesiologist as first (12.1%) or senior (12.3%) author. When considering all publications, 16.3% and 16.4% of articles had a US anesthesiologist as the first or senior author compared with articles for which non-US anesthesiologists were first (23.8%) or senior (20.9%) authors. The odds of having a US anesthesiologist as first or senior author compared to a non-US anesthesiologist for all publications were 0.6 (0.5-0.7) and 0.7 (0.6-0.9). The number of publications trended downward for both US anesthesiologists and non-US anesthesiologists during the study period. CONCLUSIONS: When compared to non-US anesthesiologists, US anesthesiologists had more CCM publications in anesthesiology journals and fewer publications in multidisciplinary CCM journals. The number of anesthesiology CCM publications decreased for both US and non-US anesthesiologists throughout the study period.