Performance of Multimodal Artificial Intelligence Chatbots Evaluated on Clinical Oncology Cases.

Importance: Multimodal artificial intelligence (AI) chatbots can process complex medical image and text-based information that may improve their accuracy as a clinical diagnostic and management tool compared with unimodal, text-only AI chatbots. However, the difference in medical accuracy of multimodal and text-only chatbots in addressing questions about clinical oncology cases remains to be tested. Objective: To evaluate the utility of prompt engineering (zero-shot chain-of-thought) and compare the competency of multimodal and unimodal AI chatbots to generate medically accurate responses to questions about clinical oncology cases. Design, Setting, and Participants: This cross-sectional study benchmarked the medical accuracy of multiple-choice and free-text responses generated by AI chatbots in response to 79 questions about clinical oncology cases with images. Exposures: A unique set of 79 clinical oncology cases from JAMA Network Learning accessed on April 2, 2024, was posed to 10 AI chatbots. Main Outcomes and Measures: The primary outcome was medical accuracy evaluated by the number of correct responses by each AI chatbot. Multiple-choice responses were marked as correct based on the ground-truth, correct answer. Free-text responses were rated by a team of oncology specialists in duplicate and marked as correct based on consensus or resolved by a review of a third oncology specialist. Results: This study evaluated 10 chatbots, including 3 multimodal and 7 unimodal chatbots. On the multiple-choice evaluation, the top-performing chatbot was chatbot 10 (57 of 79 [72.15%]), followed by the multimodal chatbot 2 (56 of 79 [70.89%]) and chatbot 5 (54 of 79 [68.35%]). On the free-text evaluation, the top-performing chatbots were chatbot 5, chatbot 7, and the multimodal chatbot 2 (30 of 79 [37.97%]), followed by chatbot 10 (29 of 79 [36.71%]) and chatbot 8 and the multimodal chatbot 3 (25 of 79 [31.65%]). The accuracy of multimodal chatbots decreased when tested on cases with multiple images compared with questions with single images. Nine out of 10 chatbots, including all 3 multimodal chatbots, demonstrated decreased accuracy of their free-text responses compared with multiple-choice responses to questions about cancer cases. Conclusions and Relevance: In this cross-sectional study of chatbot accuracy tested on clinical oncology cases, multimodal chatbots were not consistently more accurate than unimodal chatbots. These results suggest that further research is required to optimize multimodal chatbots to make more use of information from images to improve oncology-specific medical accuracy and reliability.

The research activities of Ontario's large community hospitals: an updated scoping review.

BACKGROUND: Community hospitals provide the majority of patient care in Canada but traditionally do not participate in clinical research. The disconnect between where most patients receive their health care and where health research is conducted leads to decreased study recruitment, reduced generalizability of study results, and inequitable patient access to novel therapies. A scoping review of the research activities of Ontario's large community hospitals (LCHs) between 2013 and 2015 reported an annualized output of 266 publications. In the last decade, efforts have been made to engage more community hospitals in research. In this updated scoping review, we provide a snapshot of the research activities of Ontario's LCHs between 2016 and 2022, describing the number and type of research publications as well as the frequency of collaboration within and between LCHs. METHODS: Three medical databases (PubMed, Embase, and CINAHL) were searched for publications that included at least one author affiliated with one of Ontario's 47 LCHs, and for which the topic was hospital or health related. Screening and extraction occurred concurrently. RESULTS: We identified 3,719 publications from 2016 to 22 with at least one Ontario LCH-affiliated author, representing an annualized output of 531 publications. The most frequent publication type was observational study (n = 1,654; 45%), quality improvement (n = 355; 10%), systematic reviews (n = 352; 9%) and randomized controlled trials (n = 325; 9%). The most common disciplines were outpatient care (n = 1,144; 31%), health systems research (n = 806; 22%), inpatient care (n = 437; 12%) and surgery (n = 403; 11%). LCH-affiliated first authors were identified in 997 (27%) publications, representing 755 unique authors, while LCH-affiliated senior authors were identified in 962 (26%) publications, representing 583 unique authors. Among the 1,565 studies with an LCH-affiliated first or senior author, 574 (37%) included collaborators from the same LCH and 86 (5%) included collaborators from other Ontario LCHs. CONCLUSIONS: Health research by LCH-affiliated clinicians and researchers increased significantly in 2016-2022 relative to 2013-2015. Participation in randomized controlled trials however, remains low, suggesting that further efforts are required to build clinical research infrastructure in LCHs.

Closing the gap on healthcare quality for equity-deserving groups: a scoping review of equity-focused quality improvement interventions in medicine.

INTRODUCTION: Quality improvement (QI) efforts are critical to promoting health equity and mitigating disparities in healthcare outcomes. Equity-focused QI (EF-QI) interventions address the unique needs of equity-deserving groups and the root causes of disparities. This scoping review aims to identify themes from EF-QI interventions that improve the health of equity-deserving groups, to serve as a resource for researchers embarking on QI. METHODS: In adherence with Preferred Reporting Items for Systematic Reviews and Meta-analysis guidelines, several healthcare and medical databases were systematically searched from inception to December 2022. Primary studies that report results from EF-QI interventions in healthcare were included. Reviewers conducted screening and data extraction using Covidence. Inductive thematic analysis using NVivo identified key barriers to inform future EF-QI interventions. RESULTS: Of 5,330 titles and abstracts screened, 36 articles were eligible for inclusion. They reported on EF-QI interventions across eight medical disciplines: primary care, obstetrics, psychiatry, paediatrics, oncology, cardiology, neurology and respirology. The most common focus was racialised communities (15/36; 42%). Barriers to EF-QI interventions included those at the provider level (training and supervision, time constraints) and institution level (funding and partnerships, infrastructure). The last theme critical to EF-QI interventions is sustainability. Only six (17%) interventions actively involved patient partners. DISCUSSION: EF-QI interventions can be an effective tool for promoting health equity, but face numerous barriers to success. It is unclear whether the demonstrated barriers are intrinsic to the equity focus of the projects or can be generalised to all QI work. Researchers embarking on EF-QI work should engage patients, in addition to hospital and clinic leadership in the design process to secure funding and institutional support, improving sustainability. To the best of our knowledge, no review has synthesised the results of EF-QI interventions in healthcare. Further studies of EF-QI champions are required to better understand the barriers and how to overcome them.

Mesenchymal stem cells for chronic knee pain secondary to osteoarthritis: A systematic review and meta-analysis of randomized trials.

OBJECTIVE: To assess the effectiveness of mesenchymal stem cells (MSCs) for chronic knee pain secondary to osteoarthritis (OA). METHODS: We searched MEDLINE, EMBASE, CINAHL, and Cochrane Central to September 2023 for trials that (1) enrolled patients with chronic pain associated with knee OA, and (2) randomized them to MSC therapy vs. placebo or usual care. We performed random-effects meta-analysis and used Grading of Recommendations, Assessment, Development, and Evaluation to assess the certainty of evidence. RESULTS: We included 16 trials (807 participants). At 3-6 months, MSC therapy probably results in little to no difference in pain relief (weighted mean difference [WMD] -0.74 cm on a 10 cm visual analog scale [VAS], 95% confidence interval [95%CI] -1.16 to -0.33; minimally important difference [MID] 1.5 cm) or physical functioning (WMD 2.23 points on 100-point 36-item Short Form Survey (SF-36) physical functioning subscale, 95%CI -0.97 to 5.43; MID 10-points; both moderate certainty). At 12 months, injection of MSCs probably results in little to no difference in pain (WMD -0.73 cm on a 10 cm VAS, 95%CI -1.69 to 0.24; moderate certainty) and may improve physical functioning (WMD 19.36 points on 100-point SF-36 PF subscale, 95%CI -0.19 to 38.9; low certainty). MSC therapy may increase risk of any adverse events (risk ratio [RR] 2.67, 95%CI 1.19 to 5.99; low certainty) and pain and swelling of the knee joint (RR 1.58, 95%CI 1.04 to 2.38; low certainty). CONCLUSIONS: Intra-articular injection of MSCs for chronic knee pain associated with OA probably provides little to no improvement in pain or physical function.

Transition Needs Among Veterans Living With Chronic Pain: A Systematic Review.

INTRODUCTION: A third of Canadian Armed Forces veterans report difficulty adjusting to post-military life. Moreover, an estimated 40% of Canadian veterans live with chronic pain, which is likely associated with greater needs during the transition from military to civilian life. This review explores challenges and transition needs among military personnel living with chronic pain as they return to civilian life. METHODS: We searched MEDLINE, EMBASE, CINAHL, Scopus, and Web of Science from inception to July 2022, for qualitative, observational, and mixed-method studies exploring transition needs among military veterans released with chronic pain. Reviewers, working independently and in duplicate, conducted screening and used a standardized and pilot-tested data collection form to extract data from all included studies. Content analysis was used to create a coding template to identify patterns in challenges and unmet needs of veterans transitioning to civilian life, and we summarized our findings in a descriptive manner. RESULTS: Of 10,532 unique citations, we identified 43 studies that reported transition challenges and needs of military personnel; however, none were specific to individuals released with chronic pain. Most studies (41 of 43; 95%) focused on military personnel in general, with one study enrolling individuals with traumatic brain injury and another including homeless veterans. We identified military-to-civilian challenges in seven areas: (1) identity, (2) interpersonal interactions/relationships, (3) employment, (4) education, (5) finances, (6) self-care and mental health, and (7) accessing services and care. CONCLUSIONS: Military personnel who transition to civilian life report several important challenges; however, the generalizability to individuals released with chronic pain is uncertain. Further research is needed to better understand the transition experiences of veterans with chronic pain to best address their needs and enhance their well-being.

Patients Response to Interventional Care for Chronic Pain Study (PRICS): A Cross-Sectional Survey of Community-Based Pain Clinics in Ontario, Canada.

BACKGROUND: Non-image guided injection treatments ("nerve blocks") are commonly provided in community pain clinics in Ontario for chronic non-cancer pain (CNCP) but remain controversial. AIM: We explored patients' perspectives of nerve blocks for CNCP. METHODS: We administered a 33-item cross-sectional survey to patients living with CNCP pain attending four community-based pain clinics in Ontario, Canada. The survey captured demographic information and asked about patient experiences with nerve blocks. RESULTS: Among 616 patients that were approached, 562 (91%) provided a completed survey. The mean age of respondents was 53 (SD 12), 71% were female, and the majority (57%) reported living with CNCP for more than a decade. Fifty-eight percent had been receiving nerve blocks for their pain for >3 years, 51% on a weekly frequency. Since receiving nerve blocks, patients self-reported a median improvement in pain intensity of 2.5 points (95% CI -2.5 to -3.0) on an 11-point numeric rating scale and 66% reported stopping or reducing prescription medications, including opioids. The majority who were not retired (62%) were receiving disability benefits and were unable to work in any capacity. When asked what impact cessation of nerve blocks would have, most employed patients (52%) reported they would be unable to work, and the majority indicated their ability to function across multiple domains would decrease. CONCLUSION: Our respondents who received nerve blocks for CNCP attribute important pain relief and functional improvement to this intervention. Randomized trials and clinical practice guidelines are urgently needed to optimize the evidence-based use of nerve blocks for CNCP.

Health measurement instruments and their applicability to military veterans: a systematic review.

INTRODUCTION: Accurate measurement of health status is essential to assess veterans' needs and the effects of interventions directed at improving veterans' well-being. We conducted a systematic review to identify instruments that measure subjective health status, considering four components (ie, physical, mental, social or spiritual well-being). METHODS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses, we searched CINAHL, MEDLINE, Embase, PsycINFO, Web of Science, JSTOR, ERIC, Social Sciences Abstracts and ProQuest in June 2021 for studies reporting on the development or evaluation of instruments measuring subjective health among outpatient populations. We assessed risk of bias with the Consensus-based Standards for the Selection of Health Measurement Instruments tool and engaged three veteran partners to independently assess the clarity and applicability of identified instruments. RESULTS: Of 5863 abstracts screened, we identified 45 eligible articles that reported health-related instruments in the following categories: general health (n=19), mental health (n=7), physical health (n=8), social health (n=3) and spiritual health (n=8). We found evidence for adequate internal consistency for 39 instruments (87%) and good test-retest reliability for 24 (53%) instruments. Of these, our veteran partners identified five instruments for the measurement of subjective health (Military to Civilian Questionnaire (M2C-Q), Veterans RAND 36-Item Health Survey (VR-36), Short Form 36, Abbreviated World Health Organization Quality of Life questionnaire (WHOQOL-BREF) and Sleep Health Scale) as clear and very applicable to veterans. Of the two instruments developed and validated among veterans, the 16-item M2C-Q considered most components of health (mental, social and spiritual). Of the three instruments not validated among veterans, only the 26-item WHOQOL-BREF considered all four components of health. CONCLUSION: We identified 45 health measurement instruments of which, among those reporting adequate psychometric properties and endorsed by our veteran partners, 2 instruments showed the most promise for measurement of subjective health. The M2C-Q, which requires augmentation to capture physical health (eg, the physical component score of the VR-36), and the WHOQOL-BREF, which requires validation among veterans.

Evaluation of the quality of online patient information at the intersection of complementary and alternative medicine and hypertension.

BACKGROUND: Hypertension impacts 1.1 billion people globally; many patients seek complementary and alternative medicine (CAM), as a result of adverse side effects from antihypertensive medications or because they believe natural options are safer. The internet is increasingly playing a role in patient health information-seeking behavior, however, the variability of information quality across websites is unclear. Thus, the purpose of this study was to assess the quality of websites providing consumer health information at the intersection of hypertension and CAM. METHODS: Four unique terms were searched on Google, across Australia, Canada, the US, and the UK. The first 20 webpages resulting from each search were screened for eligibility, and were included if they contained consumer health information relating to CAM therapies for the treatment/management of hypertension. To assess the quality of health information on eligible websites, we used the DISCERN instrument, a standardized quality index of consumer health information. RESULTS: Of 90 unique webpages, 40 websites were deemed eligible and quality assessed. The 40 eligible websites were classified into seven categories: professional (n = 15), news (n = 11), non-profit (n = 5), health portal (n = 3), commercial (n = 2), government (n = 1), and other (n = 3). The mean summed DISCERN score was 52.34 (standard deviation [SD] = 10.69) out of 75 and the mean overall score was 3.49 (SD = 0.08) out of 5. A total of 10 websites had a total DISCERN score of 60.00 and above with an average rating of 4.33. Among these, Medicine Net (69.00) and WebMD (69.00) were determined to have the highest quality information. Websites generally scored well with respect to providing their aims, identifying treatment benefits and options, and discussing shared-decision making; websites generally lacked references and provided inadequate information surrounding treatment risks and impact on quality of life. CONCLUSIONS: While some websites provided high-quality consumer health information, many others provided information of suboptimal quality. A need exists to better educate patients about identifying misinformation online. Healthcare providers should also inquire about their patients' health information-seeking behavior, and provide them with the guidance necessary to identify high-quality resources which they can use to inform shared-decision making.

Selective serotonin reuptake inhibitors and serotonin-norepinephrine reuptake inhibitors as adjuncts for postoperative pain management: systematic review and meta-analysis of randomised controlled trials.

BACKGROUND: We conducted a systematic review and meta-analysis to assess effects of selective serotonin reuptake inhibitors (SSRIs) and serotonin-norepinephrine reuptake inhibitors (SNRIs) as adjuncts for postoperative pain management. METHODS: We searched seven databases and two trial registers from inception to February 2021 for RCTs that compared SSRIs or SNRIs with placebo or an active control for postoperative pain management. RESULTS: We included 24 RCTs with 2197 surgical patients (21 trials for SNRIs and three trials for SSRIs). Moderate-quality evidence found that, compared with placebo, SSRIs/SNRIs (majority SNRIs) significantly reduced postoperative pain within 6 h {weighted mean difference (WMD) -0.73 cm on a 10 cm VAS (95% confidence interval [CI]: -1.04 to -0.42)}, 12 h (-0.68 cm [-1.28 to -0.07]), 24 h (-0.68 cm [-1.16 to -0.20]), 48 h (-0.73 cm [-1.22 to -0.23]), 10 days to 1 month (-0.71 cm [-1.11 to -0.31]), 3 months (-0.64 cm [-1.05 to -0.22]), and 6 months (-0.95 cm [-1.64 to -0.25]), and opioid consumption within 24 h (WMD -12 mg [95% CI: -16 to -8]) and 48 h (-10 mg [-15 to -5]), and improved patient satisfaction (WMD 0.49 point on a 1-4 Likert scale [95% CI: 0.09 to 0.89]) without significant increase in adverse events. Selective serotonin reuptake inhibitors tended to be less effective despite non-significant subgroup effects. CONCLUSIONS: Serotonin-norepinephrine reuptake inhibitors as an adjunct to standard perioperative care probably provide small reduction in both acute and chronic postoperative pain and opioid consumption, and small improvement in patient satisfaction without increases in adverse events. The effects of SSRIs are inconclusive because of very limited evidence.

Pain management program outcomes in veterans with chronic pain and comparison with nonveterans.

Background: In Canada, 41% of veterans experience chronic pain compared to the general population (20%). Many veterans with chronic pain also have comorbid disorders such as depression and posttraumatic stress disorder (PTSD), causing increased pain interference and disability. Aim: This study aims to investigate the effectiveness of a 4-week interdisciplinary pain management program at the Michael G. DeGroote Pain Clinic in Hamilton, Ontario, Canada, and to explore differences in pain experience and treatment outcomes between veterans and nonveterans in the program. Methods: Data were obtained from psychometric measures completed by 68 veterans and 68 nonveterans enrolled in the pain management program. By matching groups for age and gender, scores were compared between veterans and nonveterans. Outcomes investigated include catastrophizing, pain traumatization, stages of change, acceptance of pain, and program satisfaction. Multivariate analysis of variance (MANOVA) was conducted to examine session (admission-discharge) and group (veteran-nonveteran) differences, and independent t tests were used to examine differences in satisfaction measures. Results: Results showed that the program was effective for all participants, with significant differences between admission and discharge on several measures. However, veterans experienced significantly greater improvements in pain catastrophizing, kinesiophobia, pain traumatization, pain acceptance, stages of change, and pain coping, compared to nonveterans (P < 0.05). Though no significant differences in program satisfaction were found between groups, case managers evaluated veterans as having achieved greater benefits from the program. Conclusion: This study presents evidence supporting the effectiveness of an interdisciplinary pain management program in addressing pain-related variables in veterans and nonveterans and provides insight into how pain management is experienced differently by veterans.

Contexte: Au Canada, 41 % des anciens combattants souffrent de douleur chronique, comparativement à 20 % au sein de la population en général. De nombreux anciens combattants souffrant de douleur chronique sont également atteints de troubles comorbides tels que la dépression et le syndrome de stress post-traumatique, qui entraînent une augmentation de l'interférence de la douleur et de l'incapacité.Objectif: Cette étude porte sur l'efficacité d'un programme de prise en charge interdisciplinaire de la douleur d'une durée de quatre semaines à la Clinique de la douleur Michael G. DeGroote à Hamilton, Ontario, Canada, ainsi que sur les différences dans l'expérience de la douleur et les résultats des traitements entre les anciens combattants et d'autres patients qui n'étaient pas des anciens combattants inscrits au programme. Méthodes: Les données ont été obtenues à partir de mesures psychométriques recueillies auprés de 68 anciens combattants et de 68 patients qui n'étaient pas des anciens combattants, tous inscrits au programme de prise en charge de la douleur. En faisant correspondre les groupes selon l'âge et le sexe, les scores obtenus par les anciens combattants ont été comparés à ceux des autres patients qui n'étaient pas des anciens combattants. Les résultats étudiés comprenaient la catastrophisation, le traumatisme lié à la douleur, les étapes du changement, l'acceptation de la douleur et la satisfaction à l'égard du programme. Une analyse multivariée de la variance (MANOVA) a été effectuée pour examiner les différences entre les sessions (admission-sortie) et entre les groupes (anciens combattants et patients qui n'étaient pas des anciens combattants), et des tests t indépendants ont été utilisés pour examiner les diff'rences dans les mesures de satisfaction.Résultats: Les résultats ont montré que le programme était efficace pour tous les participants, et que des différences entre l'admission et la sortie étaient observées pour plusieurs mesures. Cependant, les anciens combattants ont connu des améliorations nettement plus importantes en ce qui concerne la catastrophisation de la douleur, la kinésiophobie, le traumatisme lié à la douleur, l'acceptation de la douleur, les stades de changement et la prise en charge de la douleur, comparativement aux patients qui n'étaient pas des anciens combattants (P < 0,05). Bien qu'aucune différence significative dans la satisfaction à l'égard du programme n'ait été constatée entre les groupes, les chargés de cas ont évalué que les anciens combattants avaient tiré plus d'avantages du programme.Conclusions: Cette étude présente des preuves de l'efficacité d'un programme interdisciplinaire de lutte contre la douleur en abordant les variables liées à la douleur chez les anciens combattants et chez des patients qui n'étaient pas des anciens combattants. Elle donne aussi un aperçu de la maniére dont la prise en charge de la douleur est vécue différemment par les anciens combattants.