Accuracy and Reliability of Chatbot Responses to Physician Questions.

Importance: Natural language processing tools, such as ChatGPT (generative pretrained transformer, hereafter referred to as chatbot), have the potential to radically enhance the accessibility of medical information for health professionals and patients. Assessing the safety and efficacy of these tools in answering physician-generated questions is critical to determining their suitability in clinical settings, facilitating complex decision-making, and optimizing health care efficiency. Objective: To assess the accuracy and comprehensiveness of chatbot-generated responses to physician-developed medical queries, highlighting the reliability and limitations of artificial intelligence-generated medical information. Design, Setting, and Participants: Thirty-three physicians across 17 specialties generated 284 medical questions that they subjectively classified as easy, medium, or hard with either binary (yes or no) or descriptive answers. The physicians then graded the chatbot-generated answers to these questions for accuracy (6-point Likert scale with 1 being completely incorrect and 6 being completely correct) and completeness (3-point Likert scale, with 1 being incomplete and 3 being complete plus additional context). Scores were summarized with descriptive statistics and compared using the Mann-Whitney U test or the Kruskal-Wallis test. The study (including data analysis) was conducted from January to May 2023. Main Outcomes and Measures: Accuracy, completeness, and consistency over time and between 2 different versions (GPT-3.5 and GPT-4) of chatbot-generated medical responses. Results: Across all questions (n = 284) generated by 33 physicians (31 faculty members and 2 recent graduates from residency or fellowship programs) across 17 specialties, the median accuracy score was 5.5 (IQR, 4.0-6.0) (between almost completely and complete correct) with a mean (SD) score of 4.8 (1.6) (between mostly and almost completely correct). The median completeness score was 3.0 (IQR, 2.0-3.0) (complete and comprehensive) with a mean (SD) score of 2.5 (0.7). For questions rated easy, medium, and hard, the median accuracy scores were 6.0 (IQR, 5.0-6.0), 5.5 (IQR, 5.0-6.0), and 5.0 (IQR, 4.0-6.0), respectively (mean [SD] scores were 5.0 [1.5], 4.7 [1.7], and 4.6 [1.6], respectively; P = .05). Accuracy scores for binary and descriptive questions were similar (median score, 6.0 [IQR, 4.0-6.0] vs 5.0 [IQR, 3.4-6.0]; mean [SD] score, 4.9 [1.6] vs 4.7 [1.6]; P = .07). Of 36 questions with scores of 1.0 to 2.0, 34 were requeried or regraded 8 to 17 days later with substantial improvement (median score 2.0 [IQR, 1.0-3.0] vs 4.0 [IQR, 2.0-5.3]; P < .01). A subset of questions, regardless of initial scores (version 3.5), were regenerated and rescored using version 4 with improvement (mean accuracy [SD] score, 5.2 [1.5] vs 5.7 [0.8]; median score, 6.0 [IQR, 5.0-6.0] for original and 6.0 [IQR, 6.0-6.0] for rescored; P = .002). Conclusions and Relevance: In this cross-sectional study, chatbot generated largely accurate information to diverse medical queries as judged by academic physician specialists with improvement over time, although it had important limitations. Further research and model development are needed to correct inaccuracies and for validation.

Long-term outcomes and persistent toxicities following BRAF/MEK inhibitor therapy for advanced melanoma.

BACKGROUND: Recent studies have shown that approximately 20% of patients have 4-5 year progression free survival (PFS) on BRAF/MEK inhibitors. The long-term safety and efficacy in these patients with more durable responses have not been studied. METHODS: This retrospective multicenter cohort study assessed response, progression, and adverse events in patients from eight institutions in four countries with >4-year PFS following BRAF/MEK inhibitors. RESULTS: Among 146 patients, 112 (76.7%) remained progression-free at median follow-up of 7.8 years from treatment start; 131 (89.7%) were alive. Among progressors (n = 34), 21 (62%) were on treatment at progression. Among those who discontinued treatment for reasons other than progression (toxicity, preference, etc.) (n = 68, with median 49 months treatment duration), 13 (19%) progressed (median 15.3 months from treatment cessation to progression). Surgery or radiation for single-organ progression resulted in durable benefit in 11 of 22 patients (50%). Subsequent systemic therapy included immune therapy (24% responded) and BRAF/MEK rechallenge (56% responded). Thirteen (8.9%) patients had ongoing toxicities at last follow-up, 10 (77%) of which remained on active treatment; all cardiac adverse events had resolved (n = 9). Twenty-four (16.4%) patients developed any new primary cancer, and 28 (19%) patients experienced other major health events. CONCLUSIONS: Over 75% of patients with 4-year PFS from BRAF/MEK inhibitors had continued durable antitumor responses after nearly 8-year median follow-up, with similar results in patients who discontinued therapy for reasons other than progression. Long-term toxicities were uncommon and low-grade. These findings highlight the often-favourable outcomes in patients with extended benefit from BRAF/MEK inhibitors.

Public Health Risks, Dermatological Manifestations, and Environmental Justice Associated With Vinyl Chloride Exposure: Narrative Review.

BACKGROUND: Environmental vinyl chloride (VC) exposure may result in serious acute and chronic dermatological conditions. Because existing literature largely focuses on exposures in occupational settings, a gap persists in our understanding of the medical consequences of large-scale chemical spills. OBJECTIVE: This study aims to examine the potential dermatological manifestations of VC exposure in the context of industrial spills and other environmental disasters and to highlight the public health and justice implications of such releases. METHODS: In this narrative review, relevant evidence-based, peer-reviewed scientific sources, gray literature, and media reports were identified via searches of search PubMed and Google using predetermined keyword search terms related to VC, VC spills and releases, train derailment, cutaneous disease, public health, and vulnerable and marginalized populations. RESULTS: Contact dermatitis and frostbite may arise acutely, highlighting the importance of swift decontamination. Long-term manifestations from chronic VC exposure due to persistence in environmental reservoirs include Raynaud disease, sclerodermatous skin changes, acro-osteolysis, and cutaneous malignancies. The clinical severity of cutaneous manifestations is influenced by individual susceptibility as well as duration, intensity, and route of exposure. Additionally, chemical releases of VC more frequently impact Communities of Color and those of lower socioeconomic status, resulting in greater rates of exposure-related disease. CONCLUSIONS: With environmental release events of hazardous chemicals becoming increasingly common and because the skin has increased contact with environmental toxins relative to other organs, an urgent need exists for a greater understanding of the overall short- and long-term health impacts of large-scale, toxic exposures, underscoring the need for ongoing clinical vigilance. Dermatologists and public health officials should also aim to better understand the ways in which the disproportionate impacts of hazardous chemical exposures on lower-income and minority populations may exacerbate existing health disparities. Herein, we describe the health implications of toxic releases with particular consideration paid to marginalized and vulnerable populations. In addition to legal and regulatory frameworks, we advocate for improved public health measures, to not only mitigate the risk of environmental catastrophes in the future, but also ensure timely and effective responses to them.

Extended Follow-Up of Chronic Immune-Related Adverse Events Following Adjuvant Anti-PD-1 Therapy for High-Risk Resected Melanoma.

Importance: Anti-programmable cell death-1 (anti-PD-1) improves relapse-free survival when used as adjuvant therapy for high-risk resected melanoma. However, it can lead to immune-related adverse events (irAEs), which become chronic in approximately 40% of patients with high-risk melanoma treated with adjuvant anti-PD-1. Objective: To determine the incidence, characteristics, and long-term outcomes of chronic irAEs from adjuvant anti-PD-1 therapy. Design, Setting, and Participants: This retrospective multicenter cohort study analyzed patients treated with adjuvant anti-PD-1 therapy for advanced and metastatic melanoma between 2015 and 2022 from 6 institutions in the US and Australia with at least 18 months of evaluable follow-up after treatment cessation (range, 18.2 to 70.4 months). Main Outcomes and Measures: Incidence, spectrum, and ultimate resolution vs persistence of chronic irAEs (defined as those persisting at least 3 months after therapy cessation). Descriptive statistics were used to analyze categorical and continuous variables. Kaplan-Meier curves assessed survival, and Wilson score intervals were used to calculate CIs for proportions. Results: Among 318 patients, 190 (59.7%) were male (median [IQR] age, 61 [52.3-72.0] years), 270 (84.9%) had a cutaneous primary, and 237 (74.5%) were stage IIIB or IIIC at presentation. Additionally, 226 patients (63.7%) developed acute irAEs arising during treatment, including 44 (13.8%) with grade 3 to 5 irAEs. Chronic irAEs, persisting at least 3 months after therapy cessation, developed in 147 patients (46.2%; 95% CI, 0.41-0.52), of which 74 (50.3%) were grade 2 or more, 6 (4.1%) were grade 3 to 5, and 100 (68.0%) were symptomatic. With long-term follow-up (median [IQR], 1057 [915-1321] days), 54 patients (36.7%) experienced resolution of chronic irAEs (median [IQR] time to resolution of 19.7 [14.4-31.5] months from anti-PD-1 start and 11.2 [8.1-20.7] months from anti-PD-1 cessation). Among patients with persistent irAEs present at last follow-up (93 [29.2%] of original cohort; 95% CI, 0.25-0.34); 55 (59.1%) were grade 2 or more; 41 (44.1%) were symptomatic; 24 (25.8%) were using therapeutic systemic steroids (16 [67%] of whom were on replacement steroids for hypophysitis (8 [50.0%]) and adrenal insufficiency (8 [50.0%]), and 42 (45.2%) were using other management. Among the 54 patients, the most common persistent chronic irAEs were hypothyroid (38 [70.4%]), arthritis (18 [33.3%]), dermatitis (9 [16.7%]), and adrenal insufficiency (8 [14.8%]). Furthermore, 54 [17.0%] patients experienced persistent endocrinopathies, 48 (15.1%) experienced nonendocrinopathies, and 9 (2.8%) experienced both. Of 37 patients with chronic irAEs who received additional immunotherapy, 25 (67.6%) experienced no effect on chronic irAEs whereas 12 (32.4%) experienced a flare in their chronic toxicity. Twenty patients (54.1%) experienced a distinct irAE. Conclusions and Relevance: In this cohort study of 318 patients who received adjuvant anti-PD-1, chronic irAEs were common, affected diverse organ systems, and often persisted with long-term follow-up requiring steroids and additional management. These findings highlight the likelihood of persistent toxic effects when considering adjuvant therapies and need for long-term monitoring and management.

Biomarkers of immune checkpoint inhibitor response and toxicity: Challenges and opportunities.

Immune checkpoint inhibitors have transformed cancer therapy, but their optimal use is still constrained by lack of response and toxicity. Biomarkers of response may facilitate drug development by allowing appropriate therapy selection and focusing clinical trial enrollment. However, aside from PD-L1 staining in a subset of tumors and rarely mismatch repair deficiency, no biomarkers are routinely used in the clinic. In addition, severe toxicities may cause severe morbidity, therapy discontinuation, and even death. Here, we review the state of the field with a focus on our research in therapeutic biomarkers and toxicities from immune checkpoint inhibitors.

Therapeutic Responses to Combination Nivolumab and Temozolomide as Salvage Therapy for Metastatic Melanoma: A Case Series.

The management of metastatic melanoma patients that fail multiple lines of systemic therapy remains a significant challenge. There is limited literature regarding combination of anti-PD-1 and temozolomide, or of other chemotherapy agents, in melanoma. Here, we present a series of 3 patients with metastatic melanoma and their responses to nivolumab and temozolomide combination therapy after progression on several local/regional therapies, combination immune checkpoint inhibitors, and/or targeted therapies. The novel combinatory strategy led to remarkable responses in all 3 patients shortly after initiating treatment with tumor remission and symptomatic improvement. The first patient has had ongoing response 15 months after initiating treatment, although he has since discontinued temozolomide due to intolerance. The remaining 2 patients show ongoing response after 4 months, with good tolerability. This case series suggests that nivolumab and temozolomide may be a promising option in the setting of advanced melanoma refractory to standard treatments, and warrants further investigation in larger series.

On the cusp: Considering the impact of artificial intelligence language models in healthcare.

Goodman et al. discuss how AI technologies like the natural language processing model Chat-GPT could potentially transform healthcare through knowledge dissemination and personalized patient education. Before these tools can be safely integrated into healthcare, research and development of robust oversight mechanisms are necessary to ensure their accuracy and reliability.

Corticosteroids and Cancer Immunotherapy.

Despite revolutionizing cancer management, immunotherapies dysregulate the immune system, leading to immune-mediated adverse events. These common and potentially dangerous toxicities are often treated with corticosteroids, which are among the most prescribed drugs in oncology for a wide range of cancer and noncancer indications. While steroids exert several mechanisms to reduce immune activity, immunotherapies, such as immune checkpoint inhibitors (ICI), are designed to enhance the immune system's inherent antitumor activity. Because ICI requires an intact and robust immune response, the immunosuppressive properties of steroids have led to a widespread concern that they may interfere with antitumor responses. However, the existing data of the effect of systemic steroids on immunotherapy efficacy remain somewhat conflicted and unclear. To inform clinical decision-making and improve outcomes, we review the impact of steroids on antitumor immunity, recent advances in the knowledge of their impact on ICI efficacy in unique populations and settings, associated precautions, and steroid-sparing treatment approaches.