ABSTRACT
What is the educational challenge? The Medical Education Research Study Quality Instrument (MERSQI) is widely used to evaluate the quality of quantitative research in medical education. It has strong evidence of validity and is endorsed by guidelines. However, the manual appraisal process is time-consuming and resource-intensive, highlighting the need for more efficient methods. What are the proposed solutions? We propose to use ChatGPT to evaluate the quality of medical education research with the MERSQI and compare its scoring with those of human evaluators. What are the potential benefits to a broader global audience? Using ChatGPT to evaluate medical education research with the MERSQI can decrease the resources required for quality appraisal. This allows faster summaries of evidence, reducing the workload of researchers, editors, and educators. Furthermore, ChatGPTs' capability to extract supporting excerpts provides transparency and may have the potential for data extraction and training new medical education researchers. What are the next steps? We plan to continue evaluating medical education research with ChatGPT using the MERSQI and other instruments to determine its feasibility in this realm. Moreover, we plan to investigate which types of studies ChatGPT performs best in.
ABSTRACT
BACKGROUND: Dengue has emerged as an unprecedented epidemic in Peru, and it is anticipated that this issue will escalate further owing to climate change. This study aimed to determine the risk factors associated with death from dengue in patients treated at Hospital II in Pucallpa, Peru. METHODOLOGY: This retrospective cohort study collected information from the medical records of patients with a diagnosis of dengue treated at Hospital II Pucallpa-Peru between January 2019 and March 2023. The primary outcome was death, and the secondary outcome was death, development of severe dengue, or Intensive Care Unit (ICU) admission. Cox regression models were used to determine risk factors. FINDINGS: The clinical records of 152 patients were evaluated, with a median age of 27.5 years (interquartile range, 11-45). Among all patients, 29 (19.1%) developed severe dengue, 31 (20.4%) were admitted to the ICU, and 13 (8.6%) died during follow-up. In the survival analysis, bilirubin >1.2 mg/dL was associated with a higher risk of death aHR: 11.38 (95% CI: 1.2 106.8). Additionally, factors associated with poor prognosis included having 1 to 3 comorbidities aRR: 1.92 (1.2 to 3.2), AST ≥251 U/L aRR: 6.79 (2.2 to 21.4), history of previous dengue aRR: 1.84 (1.0 to 3.3), and fibrinogen ≥400 mg/dL aRR: 2.23 (1.2 to 4.1). SIGNIFICANCE: Elevated bilirubin was associated with death from dengue, whereas an increase in comorbidities and a history of previous dengue were related to a poor prognosis of the disease. Early identification of severe dengue would be more feasible with improved access to laboratory testing, particularly in tropical areas with a high dengue incidence.
Subject(s)
Dengue , Humans , Peru/epidemiology , Risk Factors , Male , Adult , Female , Middle Aged , Dengue/epidemiology , Dengue/mortality , Retrospective Studies , Adolescent , Young Adult , Child , Intensive Care Units , Severe Dengue/epidemiology , Severe Dengue/mortality , Prognosis , Cohort StudiesABSTRACT
OBJECTIVE: To assess quality of life and explore its associated factors in a group of patients with chronic kidney disease (CKD) undergoing hemodialysis in Peru. METHODOLOGY: We conducted a cross-sectional analysis of patients with CKD treated at two medical centers in Tacna, Peru; between July and September 2023. We conducted a survey via telephone interviews with eligible patients using the Short Form 36 (SF 36) to assess their quality of life. RESULTS: Of 257 patients with CKD undergoing hemodialysis, we successfully interviewed 207 (59.9% males, median age: 62 years, median time on hemodialysis: 3.5 years). In the context of the SF-36 assessment, the dimensions with the lowest scores were physical role (mean: 13.9), emotional role (32.2), and physical function (32.4). Regarding the SF-36 summary scores, the average scores were 42.2 in the mental health domain and 32.0 in the physical health domain. In the adjusted model, the physical health domain score was higher in males (ß = 2.7) and those with economic self-sufficiency (ß = 3.0) and lower in older adults (ß = -2.5). The score in the mental health domain was higher in those with a higher level of education (ß = 4.1), in those with economic self-sufficiency (ß = 3.8), and in those receiving care at one of the centers included (ß: 4.2). CONCLUSION: Quality of life was affected, particularly in the realms of physical and emotional well-being. Furthermore, both the physical and mental health domains tend to show lower scores among women, older individuals, those lacking economic self-sufficiency, individuals with lower educational levels, and those with comorbidities.
Subject(s)
Quality of Life , Renal Dialysis , Renal Insufficiency, Chronic , Humans , Male , Female , Renal Dialysis/psychology , Middle Aged , Peru/epidemiology , Cross-Sectional Studies , Renal Insufficiency, Chronic/therapy , Renal Insufficiency, Chronic/psychology , Aged , Adult , Surveys and Questionnaires , Mental HealthABSTRACT
INTRODUCTION: Clinical reasoning is crucial in medical practice, yet its teaching faces challenges due to varied clinical experiences, limited time, and absence from competency frameworks. Despite efforts, effective teaching methodologies remain elusive. Strategies like the One Minute Preceptor (OMP) and SNAPPS are proposed as solutions, particularly in workplace settings. SNAPPS, introduced in 2003, offers a structured approach but lacks comprehensive evidence of its effectiveness. Methodological shortcomings hinder discerning its specific effects. Therefore, a systematic review is proposed to evaluate SNAPPS' impact on clinical reasoning teaching. CONTENT: We searched PubMed, EMBASE, and CINAHL for randomized controlled trials (RCTs) comparing SNAPPS against other methods. Data selection and extraction were performed in duplicate. Bias and certainty of evidence were evaluated using Cochrane RoB-2 and GRADE approach. SUMMARY: We identified five RCTs performed on medical students and residents. Two compared SNAPPS with an active control such as One Minute Preceptor or training with feedback. None reported the effects of SNAPPS in workplace settings (Kirkpatrick Level 3) or patients (Kirkpatrick Level 4). Low to moderate certainty of evidence suggests that SNAPPS increases the total presentation length by increasing discussion length. Low to moderate certainty of evidence may increase the number of differential diagnoses and the expression of uncertainties. Low certainty of evidence suggests that SNAPPS may increase the odds of trainees initiating a management plan and seeking clarification. OUTLOOK: Evidence from this systematic review suggests that SNAPPS has some advantages in terms of clinical reasoning, self-directed learning outcomes, and cost-effectiveness. Furthermore, it appears more beneficial when used by residents than medical students. However, future research should explore outcomes outside SNAPPS-related outcomes, such as workplace or patient-related outcomes.
Subject(s)
Clinical Reasoning , Preceptorship , Randomized Controlled Trials as Topic , Humans , Clinical Competence , Teaching , Students, Medical , Education, Medical , Internship and ResidencyABSTRACT
PURPOSE: We aimed to describe the performance and evaluate the educational value of justifications provided by artificial intelligence chatbots, including GPT-3.5, GPT-4, Bard, Claude, and Bing, on the Peruvian National Medical Licensing Examination (P-NLME). METHODS: This was a cross-sectional analytical study. On July 25, 2023, each multiple-choice question (MCQ) from the P-NLME was entered into each chatbot (GPT-3, GPT-4, Bing, Bard, and Claude) 3 times. Then, 4 medical educators categorized the MCQs in terms of medical area, item type, and whether the MCQ required Peru-specific knowledge. They assessed the educational value of the justifications from the 2 top performers (GPT-4 and Bing). RESULTS: GPT-4 scored 86.7% and Bing scored 82.2%, followed by Bard and Claude, and the historical performance of Peruvian examinees was 55%. Among the factors associated with correct answers, only MCQs that required Peru-specific knowledge had lower odds (odds ratio, 0.23; 95% confidence interval, 0.09-0.61), whereas the remaining factors showed no associations. In assessing the educational value of justifications provided by GPT-4 and Bing, neither showed any significant differences in certainty, usefulness, or potential use in the classroom. CONCLUSION: Among chatbots, GPT-4 and Bing were the top performers, with Bing performing better at Peru-specific MCQs. Moreover, the educational value of justifications provided by the GPT-4 and Bing could be deemed appropriate. However, it is essential to start addressing the educational value of these chatbots, rather than merely their performance on examinations.
Subject(s)
Artificial Intelligence , Knowledge , Humans , Cross-Sectional Studies , Peru , Educational StatusABSTRACT
BACKGROUND: ChatGPT has shown impressive performance in national medical licensing examinations, such as the United States Medical Licensing Examination (USMLE), even passing it with expert-level performance. However, there is a lack of research on its performance in low-income countries' national licensing medical examinations. In Peru, where almost one out of three examinees fails the national licensing medical examination, ChatGPT has the potential to enhance medical education. OBJECTIVE: We aimed to assess the accuracy of ChatGPT using GPT-3.5 and GPT-4 on the Peruvian National Licensing Medical Examination (Examen Nacional de Medicina [ENAM]). Additionally, we sought to identify factors associated with incorrect answers provided by ChatGPT. METHODS: We used the ENAM 2022 data set, which consisted of 180 multiple-choice questions, to evaluate the performance of ChatGPT. Various prompts were used, and accuracy was evaluated. The performance of ChatGPT was compared to that of a sample of 1025 examinees. Factors such as question type, Peruvian-specific knowledge, discrimination, difficulty, quality of questions, and subject were analyzed to determine their influence on incorrect answers. Questions that received incorrect answers underwent a three-step process involving different prompts to explore the potential impact of adding roles and context on ChatGPT's accuracy. RESULTS: GPT-4 achieved an accuracy of 86% on the ENAM, followed by GPT-3.5 with 77%. The accuracy obtained by the 1025 examinees was 55%. There was a fair agreement (κ=0.38) between GPT-3.5 and GPT-4. Moderate-to-high-difficulty questions were associated with incorrect answers in the crude and adjusted model for GPT-3.5 (odds ratio [OR] 6.6, 95% CI 2.73-15.95) and GPT-4 (OR 33.23, 95% CI 4.3-257.12). After reinputting questions that received incorrect answers, GPT-3.5 went from 41 (100%) to 12 (29%) incorrect answers, and GPT-4 from 25 (100%) to 4 (16%). CONCLUSIONS: Our study found that ChatGPT (GPT-3.5 and GPT-4) can achieve expert-level performance on the ENAM, outperforming most of our examinees. We found fair agreement between both GPT-3.5 and GPT-4. Incorrect answers were associated with the difficulty of questions, which may resemble human performance. Furthermore, by reinputting questions that initially received incorrect answers with different prompts containing additional roles and context, ChatGPT achieved improved accuracy.
ABSTRACT
Objective: To describe the academic and institutional characteristics of students who took the National Medical Internship Exam (ENIM) of the Social Security Health System (EsSalud, abbreviature in Spanish) and explore their association with the score obtained. Materials and Methods: A cross-sectional study of a secondary database of medical students who took the ENIM of EsSalud in Peru on December 27, 2022. We evaluated the factors associated with the ENIM score using linear regression. Results: We evaluated 1228 medical students who took the ENIM. The median ENIM score was 12.2 (range 10.2 to 13.8), 35.9% of students passed the exam, and only 26.5% came from a University with a licensed medicine program. In the adjusted model, the ENIM score was higher in students who belonged to a university with a licensed medicine program (ß 0.5; 95% CI: 0.2 to 0.8) and those with a university-weighted average greater than or equal to 13. And the score was lower in students from the partnership (ß -1.2; -1.6 to -0.8) or associative (ß -2.0; -2.4 to -1.6) universities compared to students from national universities. Conclusion: Medical students from national universities, universities with licensed medicine programs, and a university weighted average of 13 or greater had a better performance in EsSalud's national medical internship exam.
Objetivo: Describir las características académicas e institucionales de estudiantes que rindieron el Examen Nacional de Internado Médico (ENIM) del Seguro Social de Salud (EsSalud) y explorar su asociación con el puntaje obtenido. Materiales y métodos: Estudio transversal analítico de una base de datos secundaria de estudiantes de medicina que rindieron el ENIM del EsSalud en Perú el 27 de diciembre del 2022. Evaluamos los factores asociados al puntaje del ENIM mediante regresión lineal. Resultados: Evaluamos a 1228 estudiantes de medicina que rindieron el ENIM. La mediana de puntaje en el ENIM fue de 12,2 (rango 10,2 a 13,8), el 35,9% de los estudiantes aprobó el examen y solo el 26,5% proviene de una Universidad que cuenta con el programa de medicina ya licenciado. En el modelo ajustado, el puntaje en el ENIM fue mayor en estudiantes que pertenecían a una universidad con el programa de medicina ya licenciado (ß 0,5; IC 95%: 0,2 a 0,8) y aquellos con un promedio ponderado universitario mayor o igual a 13. Y el puntaje fue menor en estudiantes de universidades societarias (ß -1,2; -1,6 a -0,8) o asociativa (ß -2,0; -2,4 a -1,6) con respecto a estudiantes de universidades nacionales. Conclusión: Los estudiantes de medicina provenientes de universidades nacionales, universidades con programa de medicina licenciados y promedio ponderado universitario mayor o igual a 13 tuvieron un mejor desempeño en el examen nacional de internado médico de EsSalud.
ABSTRACT
PURPOSE: This study aimed to identify factors that have been studied for their associations with National Licensing Examination (ENAM) scores in Peru. METHODS: A search was conducted of literature databases and registers, including EMBASE, SciELO, Web of Science, MEDLINE, Peru's National Register of Research Work, and Google Scholar. The following key terms were used: "ENAM" and "associated factors." Studies in English and Spanish were included. The quality of the included studies was evaluated using the Medical Education Research Study Quality Instrument (MERSQI). RESULTS: In total, 38,500 participants were enrolled in 12 studies. Most (11/12) studies were cross-sectional, except for one case-control study. Three studies were published in peer-reviewed journals. The mean MERSQI was 10.33. A better performance on the ENAM was associated with a higher-grade point average (GPA) (n=8), internship setting in EsSalud (n=4), and regular academic status (n=3). Other factors showed associations in various studies, such as medical school, internship setting, age, gender, socioeconomic status, simulations test, study resources, preparation time, learning styles, study techniques, test-anxiety, and self-regulated learning strategies. CONCLUSION: The ENAM is a multifactorial phenomenon; our model gives students a locus of control on what they can do to improve their score (i.e., implement self-regulated learning strategies) and faculty, health policymakers, and managers a framework to improve the ENAM score (i.e., design remediation programs to improve GPA and integrate anxiety-management courses into the curriculum).
Subject(s)
Education, Medical , Students, Medical , Humans , Peru , Case-Control Studies , Educational MeasurementABSTRACT
Introducción: El examen nacional de medicina (ENAM) es un examen de licenciamiento para ejercer la profesión médica en Perú. Su objetivo es asegurar un perfil de calidad en el egresado peruano y brindar ventajas competitivas en la adjudicación del servicio rural y de la postulación a especialidades médicas. Todos estos aspectos ponen al ENAM como un hito en la carrera profesional. Nuestro objetivo fue determinar la correlación entre el promedio ponderado universitario y el ENAM en estudiantes de una universidad del Perú, así como identificar predictores de rendimiento para el examen.Materiales y métodos.Se realizó un estudio transversal y analítico.Resultados.De 46 estudiantes, sólo se incluyó a 45. La media de la puntuación obtenida fue de 12,4. Los factores con mayor correlación fueron el promedio ponderado universitario (r = 0,64) y el desempeño del examen de progreso tomado en la misma universidad (r = 0,64); en el análisis multivariado, un promedio ponderado universitario mayor de 13 tuvo una odds ratio de 0,62 para obtener una nota aprobatoria en el ENAM.Conclusiones.El desempeño académico durante el pregrado es el mayor predictor de un desempeño favorable en el ENAM. (AU)
Introduction: The national examination of medicine (ENAM) is a licensing exam taken in Peru to practice medicine. Its main goal is to ensure quality of care from medical doctors in Peru. It provides competitive advantages in the awarding of rural service and postulation to medical specialties depending on performance. All these aspects set the ENAM as a milestone in the professional career. Our objective was to determine the correlation between the weighted university average and the ENAM in students at a Peruvian university, as well as to identify predictors of performance for the exam.Materials and methods.An analytical, cross-sectional study was carried out.Results.Of the 46 students, only 45 were included, the mean score obtained was 12.4. The factors with the highest correlation were university weighted average (r = 0.64) and progress exam performance taken at the same university (r = 0.64); in the multivariate analysis a university weighted average above 13 had an odds ratio of 0.62 to obtain a passing grade in the ENAM.Conclusions.Academic performance during undergraduate is the strongest predictor of favorable performance on the ENAM. (AU)