Harnessing ChatGPT and GPT-4 for evaluating the rheumatology questions of the Spanish access exam to specialized medical training.

The emergence of large language models (LLM) with remarkable performance such as ChatGPT and GPT-4, has led to an unprecedented uptake in the population. One of their most promising and studied applications concerns education due to their ability to understand and generate human-like text, creating a multitude of opportunities for enhancing educational practices and outcomes. The objective of this study is twofold: to assess the accuracy of ChatGPT/GPT-4 in answering rheumatology questions from the access exam to specialized medical training in Spain (MIR), and to evaluate the medical reasoning followed by these LLM to answer those questions. A dataset, RheumaMIR, of 145 rheumatology-related questions, extracted from the exams held between 2010 and 2023, was created for that purpose, used as a prompt for the LLM, and was publicly distributed. Six rheumatologists with clinical and teaching experience evaluated the clinical reasoning of the chatbots using a 5-point Likert scale and their degree of agreement was analyzed. The association between variables that could influence the models' accuracy (i.e., year of the exam question, disease addressed, type of question and genre) was studied. ChatGPT demonstrated a high level of performance in both accuracy, 66.43%, and clinical reasoning, median (Q1-Q3), 4.5 (2.33-4.67). However, GPT-4 showed better performance with an accuracy score of 93.71% and a median clinical reasoning value of 4.67 (4.5-4.83). These findings suggest that LLM may serve as valuable tools in rheumatology education, aiding in exam preparation and supplementing traditional teaching methods.

Clinical management and discontinuation of treatment in patients with recent onset rheumatoid arthritis in a rheumatology consultation.

INTRODUCTION: The treatment of Rheumatoid Arthritis (RA) has changed dramatically in recent years, especially with the use of disease modifying drugs (DMARDs). Data on the management of this disease in clinical trials are abundant, but not so in real life. The aim of our study is to describe the management of an early RA cohort in daily clinical practice, especially DMARD discontinuations and reasons. METHODS: A retrospective observational study of patients with RA diagnosed between 01/07 and 12/14 followed up to 01/17, using >1 DMARD ≥ 3 months. VARIABLES: sociodemographic, clinical, treatment, DMARD discontinuation and reason. Descriptive analysis of sociodemographic, clinical and treatment characteristics. Discontinuation incidence rate (DIR) due to survival techniques, expressed in 100 patients*year with 95% confidence interval. RESULTS: 814 patients were included with 2388 courses of treatment, 77% women, mean age 57.5 years. First course: monotherapy (92.75%), especially Methotrexate (56.06%). In later courses there was increased combined therapy and use of biologicals (mainly Etanercept). There were 1094 discontinuations (29.5 [27.8-31.3]). The DIR was higher for adverse events (15.9 [14.7-17.3]), biologicals (49.6 [43.1-57.2]) and combined therapy. The DMAR with the lowest DIR was MTX (25.8 [23.8-28.1]). CONCLUSION: Methotrexate was the most used drug, biologicals increased throughout the follow-up, the most frequent being Etanercept. The DMARD DIR was 29*100 patients per year, mainly due to adverse events. It seems to be higher in the therapies that include biologicals and combined therapies. MTX is the drug with the lowest DIR.

Clinical Management and Discontinuation of Treatment in Patients with Recent Onset Rheumatoid Arthritis in a Rheumatology Consultation. / Manejo clínico y discontinuación de tratamiento en pacientes con artritis reumatoide de inicio en una consulta de Reumatología.

INTRODUCTION: The treatment of rheumatoid arthritis has changed dramatically in recent years, especially with the use of disease modifying drugs (DMARDs). Data on the management of this disease in clinical trials are abundant, but not so in real life. The aim of our study is to describe the management of an early rheumatoid arthritis cohort in daily clinical practice, especially DMARD discontinuations and reasons. METHODS: A retrospective observational study of patients with rheumatoid arthritis diagnosed between 01/07 and 12/14 followed up to 01/17, using>1 DMARD≥3 months. VARIABLES: sociodemographic, clinical, treatment, DMARD discontinuation and reason. Descriptive analysis of sociodemographic, clinical and treatment characteristics. Discontinuation incidence rate (DIR) due to survival techniques, expressed in 100 patients/year with 95% confidence interval. RESULTS: 814 patients were included with 2,388 courses of treatment, 77% women, mean age 57.5 years. First course: monotherapy (92.75%), especially methotrexate (56.06%). In later courses there was increased combined therapy and use of biologicals (mainly etanercept). There were 1,094 discontinuations (29.5 [27.8-31.3]). The DIR was higher for adverse events (15.9 [14.7-17.3]), biologicals (49.6 [43.1-57.2]) and combined therapy. The DMAR with the lowest DIR was methotrexate (25.8 [23.8-28.1]). CONCLUSION: Methotrexate was the most used drug, biologicals increased throughout the follow-up, the most frequent being Etanercept. The DMARD DIR was 29/100 patients per year, mainly due to adverse events. It seems to be higher in the therapies that include biologicals and combined therapies. Methotrexate is the drug with the lowest DIR.

Manejo clínico y discontinuación de tratamiento en pacientes con artritis reumatoide de inicio en una consulta de Reumatología / Clinical Management and Discontinuation of Treatment in Patients with Recent Onset Rheumatoid Arthritis in a Rheumatology Consultation

Introducción: El tratamiento de la artritis reumatoide ha cambiado drásticamente en los últimos años, sobre todo con el uso de los fármacos modificadores de la enfermedad (FAME). Los datos sobre el manejo de esta enfermedad en ensayo clínico son abundantes, pero no en la vida real. El objetivo de nuestro estudio es describir el manejo de una cohorte de artritis reumatoide de inicio en práctica clínica diaria, especialmente las suspensiones de los FAME y sus causas. Métodos: Estudio observacional retrospectivo de pacientes con artritis reumatoide diagnosticados entre 01/07 y 12/14 seguidos hasta 01/17, que usaron>1 FAME≥3 meses. Variables: sociodemográficas, clínicas, tratamiento, suspensión del FAME y causa. Análisis descriptivo de las características sociodemográficas, clínicas y de tratamiento. Incidencia de suspensión (IS) por técnicas de supervivencia, expresándose en 100 pacientes/año con intervalo de confianza del 95%. Resultados: Se incluyen 814 pacientes con 2.388 cursos de tratamiento, el 77% mujeres, edad media 57,5 años. Primer curso: monoterapia (92,75%), especialmente metotrexate (56,06%). En posteriores cursos aumentan terapia combinada y uso de biológico (principalmente etanercept). Se registraron 1.094 suspensiones (29,5 [27,8-31,3]). La IS fue mayor para evento adverso (15,9 [14,7-17,3]), biológicos (49,6 [43,1-57,2]) y terapia combinada. El FAME con menor IS fue metotrexate (25,8 [23,8-28,1]). Conclusión: El metotrexate fue el fármaco más utilizado, el biológico aumentó a lo largo del seguimiento, siendo el más frecuente etanercept. La IS de los FAME fue 29/100 pacientes año, principalmente por evento adverso. Parece mayor en las terapias que incluyen biológicos y en las combinadas. El metotrexate es el fármaco con menor IS.(AU)

Introduction: The treatment of rheumatoid arthritis has changed dramatically in recent years, especially with the use of disease modifying drugs (DMARDs). Data on the management of this disease in clinical trials are abundant, but not so in real life. The aim of our study is to describe the management of an early rheumatoid arthritis cohort in daily clinical practice, especially DMARD discontinuations and reasons. Methods: A retrospective observational study of patients with rheumatoid arthritis diagnosed between 01/07 and 12/14 followed up to 01/17, using>1 DMARD≥3 months. Variables: sociodemographic, clinical, treatment, DMARD discontinuation and reason. Descriptive analysis of sociodemographic, clinical and treatment characteristics. Discontinuation incidence rate (DIR) due to survival techniques, expressed in 100 patients/year with 95% confidence interval. Results: 814 patients were included with 2,388 courses of treatment, 77% women, mean age 57.5 years. First course: monotherapy (92.75%), especially methotrexate (56.06%). In later courses there was increased combined therapy and use of biologicals (mainly etanercept). There were 1,094 discontinuations (29.5 [27.8-31.3]). The DIR was higher for adverse events (15.9 [14.7-17.3]), biologicals (49.6 [43.1-57.2]) and combined therapy. The DMAR with the lowest DIR was methotrexate (25.8 [23.8-28.1]). Conclusion: Methotrexate was the most used drug, biologicals increased throughout the follow-up, the most frequent being Etanercept. The DMARD DIR was 29/100 patients per year, mainly due to adverse events. It seems to be higher in the therapies that include biologicals and combined therapies. Methotrexate is the drug with the lowest DIR.(AU)