Surgeons vs ChatGPT: Assessment and Feedback Performance Based on Real Surgical Scenarios.

INTRODUCTION: Artificial intelligence tools are being progressively integrated into medicine and surgical education. Large language models, such as ChatGPT, could provide relevant feedback aimed at improving surgical skills. The purpose of this study is to assess ChatGPT´s ability to provide feedback based on surgical scenarios. METHODS: Surgical situations were transformed into texts using a neutral narrative. Texts were evaluated by ChatGPT 4.0 and 3 surgeons (A, B, C) after a brief instruction was delivered: identify errors and provide feedback accordingly. Surgical residents were provided with each of the situations and feedback obtained during the first stage, as written by each surgeon and ChatGPT, and were asked to assess the utility of feedback (FCUR) and its quality (FQ). As control measurement, an Education-Expert (EE) and a Clinical-Expert (CE) were asked to assess FCUR and FQ. RESULTS: Regarding residents' evaluations, 96.43% of times, outputs provided by ChatGPT were considered useful, comparable to what surgeons' B and C obtained. Assessing FQ, ChatGPT and all surgeons received similar scores. Regarding EE's assessment, ChatGPT obtained a significantly higher FQ score when compared to surgeons A and B (p = 0.019; p = 0.033) with a median score of 8 vs. 7 and 7.5, respectively; and no difference respect surgeon C (score of 8; p = 0.2). Regarding CE´s assessment, surgeon B obtained the highest FQ score while ChatGPT received scores comparable to that of surgeons A and C. When participants were asked to identify the source of the feedback, residents, CE, and EE perceived ChatGPT's outputs as human-provided in 33.9%, 28.5%, and 14.3% of cases, respectively. CONCLUSION: When given brief written surgical situations, ChatGPT was able to identify errors with a detection rate comparable to that of experienced surgeons and to generate feedback that was considered useful for skill improvement in a surgical context performing as well as surgical instructors across assessments made by general surgery residents, an experienced surgeon, and a nonsurgeon feedback expert.

Diseño, implementación y evaluación de un programa de aprendizaje de habilidades procedimentales básicas en enfermería utilizando la plataforma C1DO1 durante la pandemia por COVID-19

Introducción: La simulación remota brindó una alternativa concreta para entrenar las habilidades procedimentales básicas durante la pandemia. El objetivo de este artículo es describir el proyecto que mantuvo el entrenamiento de las habilidades procedimentales básicas de enfermería en alumnos de pregrado UC, utilizando la aplicación C1DO1, durante el segundo semestre del 2021. Materiales y Métodos: Se diseñaron cinco etapas en la plataforma C1DO1. Cada etapa consideró un procedimiento clínico básico. Los estudiantes practicaron las técnicas en su hogar y se grabaron ejecutando el procedimiento. El video fue revisado por un docente que proporcionaba una retroalimentación personalizada y medía su aprendizaje de manera formativa. Al finalizar todas las etapas, se aplicó una encuesta para evaluar la percepción y satisfacción con el proyecto. Resultados: 183 estudiantes se inscribieron en el proyecto, de los cuales 122 participaron activamente. El porcentaje de respuesta del instrumento de evaluación fue de un 71%. La satisfacción global alcanzó una nota de 6,1. El grado de acuerdo con la simulación remota fue de un 94%. La percepción de autoeficacia alcanzó un 89%. La retroalimentación personalizada fue considerada como muy importante por el 94% de los participantes y el 98% de los estudiantes aprobó la etapa que efectúo. Discusión: Existe un alto grado de acuerdo con el entrenamiento de habilidades procedimentales mediante la simulación remota, sin embargo, los estudiantes consideran esta metodología como un complemento al aprendizaje motor, aún faltan estudios que midan transferencia.

Introduction: Remote simulation provided a concrete alternative to train basic procedural skills, given the pandemic. This article aims to describe the project that maintained the training of basic nursing procedural skills in UC undergraduate students using the C1DO1 application during the second semester of 2021. Methods: Five stages were designed on the C1DO1 platform. Each stage considered a basic clinical procedure. The students practiced the techniques at home and recorded themselves performing the procedure. The video was reviewed by a teacher who provided personalized feedback and measured their learning in a formative way. At the end of all the stages, a survey was applied to the students to evaluate their perception and satisfaction with the project. Results: The total number of students enrolled in the project was 183 of which 122 actively participated. The response rate of the evaluation instrument was 71%. Global satisfaction with the project obtained a score of 6.1. The degree of agreement with the remote simulation was 94%. The perception of self-efficacy reached 89%. The personalized feedback was considered very important by 94% of the participants and 98% of the students approved the stage they carried out. Discussion: There is a high degree of agreement with the training of procedural skills through remote simulation, however, students see this methodology as a complement to motor learning, and studies that measure transfer are still lacking.

An international consensus for mitigation of the detrimental effects of the COVID-19 pandemic on laparoscopic training.

AIM: Achieve an international consensus on how to recover lost training opportunities. The results of this study will help inform future EAES guidelines about the recovery of surgical training before and after the pandemic. BACKGROUND: A global survey conducted by our team demonstrated significant disruption in surgical training during the COVID-19 pandemic. This was wide-spread and affected all healthcare systems (whether insurance based or funded by public funds) in all participating countries. Thematic analysis revealed the factors perceived by trainees as barriers to training and gave birth to four-point framework of recovery. These are recommendations that can be easily achieved in any country, with minimal resources. Their implementation, however, relies heavily on the active participation and leadership by trainers. Based on the results of the global trainee survey, the authors would like to conduct a Delphi-style survey, addressed to trainers on this occasion, to establish a pragmatic step-by-step approach to improve training during and after the pandemic. METHODS: This will be a mixed qualitative and quantitative study. Semi-structured interviews will be performed with laparoscopic trainers. These will be transcribed and thematic analysis will be applied. A questionnaire will then be proposed; this will be based on both the results of the semi structured interviews and of the global trainee survey. The questionnaire will then be validated by the steering committee of this group (achieve consensus of >80%). After validation, the questionnaire will be disseminated to trainers across the globe. Participants will be asked to consent to participate in further cycles of the Delphi process until more than 80% agreement is achieved. RESULTS: This study will result in a pragmatic framework for continuation of surgical training during and after the pandemic (with special focus on minimally invasive surgery training).