The progress test as a structuring initiative for programmatic assessment.

BACKGROUND: The Progress Test is an individual assessment applied to all students at the same time and on a regular basis. The test was structured in the medical undergraduate education of a conglomerate of schools to structure a programmatic assessment integrated into teaching. This paper presents the results of four serial applications of the progress test and the feedback method to students. METHODS: This assessment comprises 120 items offered online by means of a personal password. Items are authored by faculty, peer-reviewed, and approved by a committee of experts. The items are classified by five major areas, by topics used by the National Board of Medical Examiners and by medical specialties related to a national Unified Health System. The correction uses the Item Response Theory with analysis by the "Rasch" model that considers the difficulty of the item. RESULTS: Student participation increased along the four editions of the tests, considering the number of enrollments. The median performances increased in the comparisons among the sequential years in all tests, except for test1 - the first test offered to schools. Between subsequent years of education, 2nd-1st; 4th-3rd and 5th-4th there was an increase in median scores from progress tests 2 through 4. The final year of undergraduate showed a limited increase compared to the 5th year. There is a consistent increase in the median, although with fluctuations between the observed intervals. CONCLUSION: The progress test promoted the establishment of regular feedback among students, teachers and coordinators and paved the road to engagement much needed to construct an institutional programmatic assessment.

[Living Networks in Health during the covid-19 pandemic: Evaluation of the multiprofessional training course "Nós da Linha de Frente" in Manaus, Amazonas, Brazil]. / Redes vivas en salud en la pandemia de covid-19: evaluación del curso de formación multiprofesional "Nós da Linha de Frente" en Manaus, Amazonas, Brasil.

This article presents the evaluation of the therapeutic intervention course in situations of traumatic loss and grief, offered online by the Universidade do Estado do Amazonas, between April and August 2021, for volunteer professionals. A semi-structured questionnaire with open and closed questions was administered. The analysis of the 55 responses from volunteer professionals and 14 responses from individuals who received the intervention was based on the Kirkpatrick evaluation model. The results demonstrate the theoretical and practical success of the course, with a positive assessment at the reaction, learning, behavior, and results levels, emphasizing collective work and the motivation of the course speakers. The main acquired knowledge pertained to topics related to death and grief care. The behaviors resulting from the course fall within the affective-social and technical-professional domains. Regarding impact, there was a predominance of satisfaction among the individuals who received the intervention, as well as identification of changes brought about by the training.

Este artículo presenta la evaluación del curso de intervención terapéutica en situaciones de pérdida traumática y duelo, ofrecido en línea por la Universidade do Estado do Amazonas, entre abril y agosto de 2021, para profesionales voluntarios. Se aplicó un cuestionario semiestructurado con preguntas abiertas y cerradas. El análisis de las 55 respuestas de profesionales voluntarios y 14 respuestas de personas atendidas se basó en el modelo de evaluación Kirkpatrick. Los resultados muestran el acierto teórico y práctico del curso, con una evaluación positiva en los niveles de reacción, aprendizaje, comportamiento y resultados, en la que se destacó el trabajo colectivo y la motivación de los disertantes del curso. Los principales contenidos aprehendidos fueron temas relacionados con el cuidado de la muerte y el duelo. Las conductas derivadas del curso se encuadran en los campos afectivo-social y técnico-profesional. En cuanto al impacto, hubo un predominio de la satisfacción de las personas atendidas, así como identificación de cambios provocados por la formación.

Una revisión sistemática del uso de la inteligencia artificial en la educación / A systematic review of the use of artificial intelligence in education

Introducción. El uso de la inteligencia artificial (IA) en la educación ha sido objeto de una creciente atención en los últimos años. La IA se ha utilizado para mejorar la personalización del aprendizaje, la retroalimentación y la evaluación de los estudiantes. Sin embargo, también hay desafíos y limitaciones asociados. El objetivo de este trabajo fue identificar las principales tendencias y áreas de aplicación de la inteligencia artificial en la educación, así como analizar los beneficios y limitaciones de su uso en este ámbito. Métodos. Se llevó a cabo una revisión sistemática que exploró el empleo de la inteligencia artificial en el ámbito educativo. Esta revisión siguió una metodología de investigación basada en la búsqueda de literatura, compuesta por cinco etapas. La investigación se realizó utilizando Scopus como fuente de consulta primaria y se empleó la herramienta VOSviewer para analizar los resultados obtenidos. Resultados. Se encontraron numerosos estudios que investigan el uso de la IA en la educación. Los resultados sugieren que la IA puede mejorar significativamente la personalización del aprendizaje, proporcionando recomendaciones de actividades y retroalimentación adaptadas a las necesidades individuales de cada estudiante. Conclusiones. A pesar de las ventajas del uso de la IA en la educación, también hay desafíos y limitaciones que deben abordarse, como la calidad de los datos utilizados por la IA, la necesidad de capacitación para educadores y estudiantes, y las preocupaciones sobre la privacidad y la seguridad de los datos de los estudiantes. Es importante seguir evaluando los efectos del uso de la IA en la educación para garantizar su uso efectivo y responsable.

Introduction. The use of artificial intelligence (AI) in education has been the subject of increasing attention in recent years. AI has been used to improve personalized learning, feedback, and student assessment. However, there are also challenges and limitations. The aim of this study was to identify the main trends and areas of application of artificial intelligence in education, as well as to analyze the benefits and limitations of its use in this field. Methods. A systematic review was carried out on the use of artificial intelligence in education, using a literature search research methodology with five stages, based on the Scopus query and the tool for analyzing results with VOSviewer. Results. Numerous studies investigating the use of AI in education were found. The results suggest that AI can significantly improve personalized learning by providing activity recommendations and feedback tailored to the individual needs of each student. Conclusions. Despite the advantages of using AI in education, there are also challenges and limitations that need to be addressed, such as the quality of data used by AI, the need for training for educators and students, and concerns about the privacy and security of student data. It is important to continue evaluating the effects of AI use in education to ensure its effective and responsible use.

Perfil e percurso profissional de egressos dos cursos de mestrado e doutorado da Fundação Oswaldo Cruz (2013-2020) / Profile and career path of graduates from the Oswaldo Cruz Foundation master's and doctoral courses (2013-2020) / Perfil y trayectoria profesional de egresados de los cursos de maestría y doctorado de la Fundación Oswaldo Cruz (2013-2020)

Resumo: O objetivo do estudo é analisar o perfil de egressos e identificar fatores associados ao desempenho positivo relativo à trajetória de formação e de inserção profissional entre aqueles que concluíram cursos presenciais de mestrado e doutorado acadêmico da Fundação Oswaldo Cruz (Brasil). Participaram do estudo 2.462 egressos (1.402 de mestrado e 1.060 de doutorado) que responderam um questionário digital, contendo questões sobre perfil do egresso e inserção profissional antes do ingresso no curso e após a conclusão. Foi criada uma medida sobre "desempenho positivo pós-curso". Os resultados revelam que egressos de mestrado com impacto positivo do curso tendem a ser homens; ter especialização antes de ingressar no curso; ter emprego remunerado após o término do curso nas áreas de pesquisa, educação, assistência e gestão; e ser servidor público. No doutorado o seguinte perfil é evidenciado: egressos com emprego remunerado após o término do curso nas áreas de pesquisa ou educação; contratados por Regime Jurídico Único ou contrato temporário de pessoa jurídica; e com publicação científica ou patente. A avaliação/acompanhamento de egressos deve se instaurar como uma importante política pública educacional, incorporada no processo de autoavaliação institucional, o que possibilita rever rumos e prioridades na agenda educacional e pedagógica da instituição.

Abstract: This study aims to analyze the profile of graduates and identify factors associated with positive performance regarding training and professional insertion among those who have completed face-to-face master's and doctoral courses at the Oswaldo Cruz Foundation (Brazil). A total of 2,462 graduates participated in the study (1,402 master's and 1,060 doctoral graduates), who answered a digital questionnaire containing questions about individual characteristics and professional insertion before entering the course and after completion. A measure on "positive post-course performance" was created. The results show that Master's graduates with a positive impact from the course tend to be men; to have a specialization degree before entering the course; to have paid employment after finishing the course in the areas of research, education, assistance, and management; and to be civil servants. In the Doctorate, the following profile is evident: graduates with paid employment after finishing the course in the areas of research or education; hired under a single legal regime or temporary contract as a legal entity; and with a scientific publication or patent. Evaluating/monitoring graduates must be established as an important public educational policy, incorporated into the institutional self-evaluation process, which makes it possible to review directions and priorities in the institution's educational and pedagogical agenda.

Resumen: El objetivo del estudio es analizar el perfil de egresados e identificar factores asociados al desempeño positivo relacionado a la trayectoria de formación e inserción profesional entre aquellos que concluyeron cursos presenciales de maestría y doctorado académico de la Fundación Oswaldo Cruz (Brasil). Participaron del estudio 2.462 egresados (1.402 de maestría y 1.060 de doctorado) que contestaron el cuestionario de manera digital, con preguntas sobre el perfil del egresado y la inserción profesional antes del ingreso al curso y tras su conclusión. Se creó una medida sobre "desempeño positivo tras el curso". Los resultados muestran que los egresados de la maestría con un impacto positivo del curso tienden a ser hombres; tener especialización antes de ingresar al curso; tener trabajo remunerado tras concluir el curso en las áreas de investigación, educación, asistencia y gestión; y ser funcionario público. Se evidencia el siguiente perfil en el doctorado: egresados con trabajo remunerado tras el término del curso en las áreas de investigación o educación; contratados bajo un régimen jurídico único o un contrato temporal de persona jurídica; y con publicación científica o patente. La evaluación/seguimiento de egresados se debe establecer como una importante política pública educativa, incorporada en el proceso de autoevaluación institucional, lo que permite revisar direcciones y prioridades en la agenda educativa y pedagógica de la institución.

Redes vivas en salud en la pandemia de covid-19: evaluación del curso de formación multiprofesional "Nós da Linha de Frente" en Manaus, Amazonas, Brasil / Living Networks in Health during the covid-19 pandemic: Evaluation of the multiprofessional training course "Nós da Linha de Frente" in Manaus, Amazonas, Brazil

RESUMEN Este artículo presenta la evaluación del curso de intervención terapéutica en situaciones de pérdida traumática y duelo, ofrecido en línea por la Universidade do Estado do Amazonas, entre abril y agosto de 2021, para profesionales voluntarios. Se aplicó un cuestionario semiestructurado con preguntas abiertas y cerradas. El análisis de las 55 respuestas de profesionales voluntarios y 14 respuestas de personas atendidas se basó en el modelo de evaluación Kirkpatrick. Los resultados muestran el acierto teórico y práctico del curso, con una evaluación positiva en los niveles de reacción, aprendizaje, comportamiento y resultados, en la que se destacó el trabajo colectivo y la motivación de los disertantes del curso. Los principales contenidos aprehendidos fueron temas relacionados con el cuidado de la muerte y el duelo. Las conductas derivadas del curso se encuadran en los campos afectivo-social y técnico-profesional. En cuanto al impacto, hubo un predominio de la satisfacción de las personas atendidas, así como identificación de cambios provocados por la formación.

ABSTRACT This article presents the evaluation of the therapeutic intervention course in situations of traumatic loss and grief, offered online by the Universidade do Estado do Amazonas, between April and August 2021, for volunteer professionals. A semi-structured questionnaire with open and closed questions was administered. The analysis of the 55 responses from volunteer professionals and 14 responses from individuals who received the intervention was based on the Kirkpatrick evaluation model. The results demonstrate the theoretical and practical success of the course, with a positive assessment at the reaction, learning, behavior, and results levels, emphasizing collective work and the motivation of the course speakers. The main acquired knowledge pertained to topics related to death and grief care. The behaviors resulting from the course fall within the affective-social and technical-professional domains. Regarding impact, there was a predominance of satisfaction among the individuals who received the intervention, as well as identification of changes brought about by the training.

Reliability across content areas in progress tests assessing medical knowledge: a Brazilian cross-sectional study with implications for medical education assessments

ABSTRACT BACKGROUND: Brazilian medical schools equitably divide their medical education assessments into five content areas: internal medicine, surgery, pediatrics, obstetrics and gynecology, and public health. However, this division does not follow international patterns and may threaten the examinations' reliability and validity. OBJECTIVE: To assess the reliability indices of the content areas of serial, cross-institutional progress test examinations. DESIGN AND SETTINGS: This was an analytical, observational, and cross-sectional study conducted at nine public medical schools (mainly from the state of São Paulo) with progress test examinations conducted between 2017 and 2023. METHODS: The examinations covered the areas of basic sciences, internal medicine, surgery, pediatrics, obstetrics and gynecology, and public health. We calculated reliability indices using Cronbach's α, which indicates the internal consistency of a test. We used simple linear regressions to analyze temporal trends. RESULTS: The results showed that the Cronbach's α for basic sciences and internal medicine presented lower values, whereas gynecology, obstetrics, and public health presented higher values. After changes in the number of items and the exclusion of basic sciences as a separate content area, internal medicine ranked highest in 2023. Individually, all content areas except pediatrics remained stable over time. CONCLUSIONS: Maintaining an equitable division in assessment content may lead to suboptimal results in terms of assessment reliability, especially for internal medicine. Therefore, content sampling of medical knowledge for general assessments should be reappraised.

Implementation of the São Paulo Nursing Courses Consortium for the Progress Test: experience report / Implementación del Consorcio de Cursos de Enfermería de São Paulo para la Prueba de Progreso: relato de experiencia / Implementação do Consórcio dos Cursos de Enfermagem Paulistas para o Teste de Progresso: relato de experiência

ABSTRACT Objective: To report the experience of implementing the São Paulo Nursing Courses Consortium for the Progress Test. Method: This is an experience report of the consortium's work in Progress Test preparation and application for Public Schools of Nursing in São Paulo in 2019, 2021 and 2022, with a descriptive analysis of the work process and the results obtained. Results: The consortium's activities are structured into the following stages: planning; theme review; distributing and requesting questions; professor training; question elaboration; question reception; question selection; question validation; student registration; test application; analysis and dissemination of results. A total of 57.3% of enrolled students participated. There was a predominance of questions of medium difficulty and a gradual progression in the level of discrimination of the questions, with, in 2022, 82.5% being considered adequate. Final considerations: The consortium has allowed the test to be applied interinstitutionally, with greater scope, accuracy, and quality of questions. Through this experience, it is expected to encourage progress testing in undergraduate nursing courses in other contexts.

RESUMEN Objetivo: Relatar la experiencia de implementación del Consorcio de Cursos de Enfermería de São Paulo para la Prueba de Progreso. Método: Se trata de un relato de experiencia del trabajo del consorcio en la elaboración y aplicación de la Prueba de Progreso para las Escuelas Públicas de Enfermería de São Paulo en los años 2019, 2021 y 2022, con un análisis descriptivo del proceso de trabajo y de los resultados obtenidos. Resultados: Las actividades del consorcio se estructuran en las siguientes etapas: planificación; revisión de temas; distribución y solicitud de preguntas; capacitación de docentes; elaboración de preguntas; recepción de preguntas; selección de preguntas; validación de preguntas; registro de estudiantes; aplicación de la prueba; análisis y difusión de resultados. Participaron el 57,3% de los estudiantes matriculados. Hubo un predominio de preguntas de dificultad media y una progresión gradual en el nivel de discriminación de las preguntas, considerándose adecuadas en 2022 un 82,5%. Consideraciones finales: El consorcio ha permitido que la prueba sea aplicada de manera interinstitucional, con mayor alcance, precisión y calidad de preguntas. A través de esta experiencia, se espera fomentar la realización de la prueba de progreso en cursos de pregrado en enfermería en otros contextos.

RESUMO Objetivo: Relatar a experiência da implementação do Consórcio dos Cursos de Enfermagem Paulistas para o Teste de Progresso. Método: Trata-se de relato de experiência da atuação do consórcio na elaboração e aplicação do Teste de Progresso para as Escolas de Enfermagem Públicas Paulistas nos anos de 2019, 2021 e 2022, com análise descritiva do processo de trabalho e dos resultados obtidos. Resultados: As atividades do consórcio estão estruturadas nas seguintes etapas: planejamento; revisão de temas; distribuição e solicitação de questões; capacitação de docentes; elaboração de questões; recepção de questões; seleção de questões; validação de questões; inscrição de estudantes; aplicação do teste; análise e divulgação de resultados. Participaram 57,3% dos estudantes matriculados. Houve predomínio de questões de média dificuldade e progressão gradual do nível de discriminação das questões, sendo que, em 2022, 82,5% foram consideradas adequadas. Considerações finais: O consórcio tem permitido a aplicação do teste de forma interinstitucional, com maior abrangência, acurácia e qualidade das questões. Por meio dessa experiência, espera-se fomentar a realização do teste de progresso em cursos de graduação em enfermagem em outros contextos.

Qualidade da residência médica na perspectiva das partes interessadas: revisão de escopo e painel Delphi / Quality of medical residency from the stakeholders' perspective: a scoping review and Delphi panel

RESUMO Introdução: Embora na literatura médica existam diversas métricas para avaliar a qualidade dos programas de residência, os instrumentos avaliativos tendem a focar as dimensões dos residentes e preceptores. Isso negligencia uma ampla gama de partes interessadas (stakeholders), cujas perspectivas são fundamentais para uma compreensão holística da qualidade dos programas. Objetivo: Diante desse contexto, esta revisão teve como objetivos mapear os principais stakeholders envolvidos na residência médica, identificar e categorizar as métricas de avaliação da qualidade dos programas mais prevalentes na literatura, e analisar a sua relevância em relação às perspectivas das partes interessadas. Método: Inicialmente, foi realizada uma revisão de escopo da literatura para identificar e categorizar os stakeholders, além de mapear as métricas de qualidade. Posteriormente, um painel Delphi foi conduzido para analisar a relevância dessas métricas em relação às perspectivas das partes interessadas identificadas. Resultado: Foram mapeados 14 stakeholders e identificadas 17 métricas, posteriormente divididas em quatro categorias principais. As métricas "adaptabilidade" e "bem-estar" se destacaram, sendo unanimemente reconhecidas por todos os stakeholders como "favorável" ou "altamente favorável". Por sua vez, "autoavaliação" e "satisfação do paciente" receberam avaliações mais cautelosas ou negativas. Os painelistas enfatizaram que "nenhuma métrica é capaz de fornecer individualmente uma avaliação precisa da qualidade de um programa de residência médica". Conclusão: Ao mapear os stakeholders da residência médica, bem como identificar, categorizar e analisar as métricas de avaliação da qualidade mais prevalentes, este estudo ampliou o debate sobre a complexidade das perspectivas em torno da formação médica. A diversidade de atores envolvidos justifica valorizações distintas das várias dimensões da qualidade, reforçando a conclusão de que métricas isoladas não capturam integralmente a qualidade dos programas. Na prática, os resultados sublinham a importância da implementação de sistemas de avaliação da qualidade que sejam equilibrados e alinhados com as expectativas e necessidades dos principais stakeholders.

ABSTRACT Introduction: Medical literature abounds with metrics to assess the quality of residency programs, yet evaluative tools predominantly focus on the dimensions relevant to residents and preceptors. This overlooks a wide range of stakeholders, whose perspectives are essential for a holistic understanding of program quality. Given this context, this review aimed to map key stakeholders in medical residency, identify and categorize the most prevalent quality assessment metrics found in the literature, and analyze their relevance in relation to the stakeholders' perspectives. Method: Initially, a scoping review of the literature was conducted to identify and categorize stakeholders, as well as to map quality metrics. This was followed by a Delphi panel to assess the relevance of these metrics in relation to the perspectives of the identified stakeholders. Results: Fourteen stakeholders were mapped, and seventeen metrics were identified, subsequently divided into four main categories. The metrics "Adaptability" and "Well-being" stood out, being unanimously recognized by all stakeholders as "Favorable" or "Highly Favorable". Conversely, "Self-assessment" and "Patient Satisfaction" elicited more cautious or negative evaluations. Panelists emphasized that "no single metric can provide an accurate assessment of the quality of a medical residency program on its own". Conclusion: By mapping the stakeholders of medical residency, as well as identifying, categorizing, and analyzing the most prevalent quality assessment metrics, this study broadened the debate on the complexity of perspectives surrounding medical education. The diversity of actors observed justifies distinct evaluations across the various dimensions of quality, reinforcing the conclusion that isolated metrics cannot fully capture the program quality. In practice, the findings underscore the importance of implementing quality assessment frameworks that are balanced and aligned with the expectations and needs of the main stakeholders.

Bridging Validity Frameworks in Assessment: Beyond Traditional Approaches in Health Professions Education.

Construct: High-stakes assessments measure several constructs, such as knowledge, competencies, and skills. In this case, validity evidence for test scores' uses and interpretations is of utmost importance, because of the consequences for everyone involved in their development and implementation. Background: Educational assessment requires an appropriate understanding and use of validity frameworks; however, health professions educators still struggle with the conceptual challenges of validity, and frequently validity analyses have a narrow focus. Important obstacles are the plurality of validity frameworks and the difficulty of grounding these abstract concepts in practice. Approach: We reviewed the validity frameworks literature to identify the main elements of frequently used models (Messick and Kane's) and proposed linking frameworks including Russell's recent overarching proposal. Examples are provided with commonly used assessment instruments in health professions education. Findings: Several elements in these frameworks can be integrated into a common approach, matching and aligning Messick's sources of validity with Kane's four inference types. Conclusions: This proposal to contribute evidence for assessment inferences may provide guidance to understanding the use of validity evidence in applied settings. The evolving field of validity research provides opportunities for its integration and practical use in health professions education.

Terminal efficiency, Lag and dropout in cohorts from 2007 to 2014 of dental students at the University of Costa Rica / Eficiencia terminal, rezago y abandono en las cohortes del 2007 al 2014 de estudiantes de Odontología de la Universidad de Costa Rica

Abstract Determine the terminal efficiency, lag and dropout in the cohorts of students who entered the dentistry career at the Faculty of Dentistry at University of Costa Rica in the lapse 2007 to 2014. Data from 736 files were collected. The variables considered were sex, admission age, nationality, marital status, children, admission note, domicile and high school. The data was collected from the Student Application System, the physical files, and the data base from the Supreme Court of Elections of Costa Rica. Descriptive statistics, bivariate and multivariate analysis were made, which were implemented from two logistic regression models. 98% of the students were Costa Rican, 68% women, 79% entered according to the admission note, 43% entered with an age of 18 years or less, 50% came from a public school, 77% resided in the Greater Metropolitan Area and 95% were single and remained without children. The average terminal efficiency was 6%; 46% of students have graduated with lag, 16% are still enrolled and 32% dropped out. Sex, age, admission note, and motherhood are sociodemographic variables that are associated with terminal efficiency and dropout. The average terminal efficiency in the cohorts from 2007 to 2014 in the courses at the Faculty of Dentistry University of Costa Rica was very low, almost half of the students graduated with lag and about a third dropped out the studies. The grade from the admission note seems to be a predictor of students'academic behavior, higher grade had more chances of graduating and less likely to dropout.

Resumen Determinar la eficiencia terminal, el rezago y el abandono en las cohortes de los estudiantes que ingresaron a la carrera de Odontología de la Facultad de Odontología UCR en el período 2007 al 2014. Se recopilaron los datos de 736 expedientes. Las variables consideradas fueron: sexo, edad de ingreso, nacionalidad, estado civil, hijos, lugar y colegio de procedencia, y nota de examen de admisión. Los datos se recopilaron del Sistema de Aplicaciones Estudiantiles, los expedientes físicos y del Tribunal Supremo de Elecciones de Costa Rica. Se realizó estadística descriptiva, análisis bivariado y multivariado que se implementó a partir de dos modelos de regresión logística. El 98% de los estudiantes fueron costarricenses, el 68% mujeres, el 79% ingresó según la nota de admisión, el 43% ingresó con una edad de 18 años o menos, el 50% provenía de un colegio público, el 77% residía en la gran área Metropolitana y el 95% eran solteros y permanecieron sin hijos. La eficiencia terminal en promedio fue de 6%; el 46% de los estudiantes se han graduado con rezago, el 16% continúan matriculados y el 32% hizo abandono de los estudios. El sexo, la edad, la nota del examen de admisión y la maternidad son variables sociodemográficas que se asocian con la eficiencia terminal y el abandono. El promedio de la eficiencia terminal en las cohortes del 2007 al 2014 en la carrera de Odontología de la Facultad de Odontología UCR fue muy bajo, casi la mitad de los estudiantes se han graduado con rezago y cerca de un tercio hizo abandono de los estudios. La nota del examen de admisión parece ser un predictor en el comportamiento académico de los estudiantes: a mayor nota más posibilidades de graduarse y menos de abandonar la carrera.

Percepciones de profesores y estudiantes acerca de la utilización del portafolios en la asignatura Prevención en Salud

Fundamento: en el ámbito de la pedagogía universitaria, el modelo docente adopta una nueva visión, el aprendizaje desarrollador y la evaluación educativa; en este contexto ha surgido, en los últimos años, el portafolios. Objetivo valorar la percepción de profesores y estudiantes sobre la utilización del portafolio como herramienta de aprendizaje en la asignatura Prevención en Salud. Métodos: investigación educacional, realizada en la Universidad de Ciencias Médicas de Cienfuegos en el primer período del curso 2022, sobre un universo constituido por el total de estudiantes de segundo año que cursaron la asignatura y los profesores que fungieron como tutores. La muestra fue seleccionada al azar, 64 estudiantes y 36 profesores, que aceptaron participar en la investigación respetando los principios éticos. Se empleó la técnica de la entrevista, se diseñó un cuestionario semiestructurado para explorar la opinión de estudiantes y profesores. Resultados: los docentes consideraron de utilidad el portafolio para el aprendizaje, al permitir objetivar y documentar las competencias, logrando dinamismo y estímulo para la reflexión; alegaron inconformidad en la evaluación refiriendo que requiere de mucho tiempo. Los estudiantes recibieron orientaciones sobre el portafolio, que les permitieron prepararse para su profesión, organizarse y autoevaluarse. Como debilidad destacaron la laboriosidad al considerarlo estresante, refiriéndose a la responsabilidad y al tiempo disponible e insuficiente motivación que brindan los tutores. Conclusiones: el portafolio es un instrumento centrado en el aprendizaje de competencias, con participación del estudiante y un tipo de evaluación que estimula la responsabilidad aplicando los conocimientos de la vida real en la práctica médica.

Foundation: in the university pedagogy field, the teaching model adopts a new vision: developer learning and educational evaluation; in this context, the portfolio has emerged in recent years. Objective: to assess the perception of teachers and students about the use of the portfolio as a learning tool in the Health Prevention subject. Methods: educational research, at the Cienfuegos Medical Sciences University in the first period of the 2022 academic year, a universe made up of the total number of second-year students who took the subject and the professors who served as tutors. The sample was randomly selected, 64 students and 36 teachers, who agreed to participate in the research respecting ethical principles. The interview technique was used, a semi-structured questionnaire was designed, exploring the opinion of students and teachers. Results: the teachers considered the portfolio useful for learning, as it allowed objectifying and documenting the competencies, achieving dynamism and stimulating reflection, they claimed disagreement in the evaluation, stating that it requires a lot of time. The students received guidance on the portfolio, which allowed them to prepare for their profession, organize and self-evaluate. As a weakness, they highlighted industriousness, considering it stressful, referring to the responsibility and the time available and insufficient motivation provided by the tutors. Conclusions: the portfolio is an instrument focused on learning competencies, with student participation and a type of evaluation that stimulates responsibility by applying real-life knowledge in medical practice.

Performance of ChatGPT, Bard, Claude, and Bing on the Peruvian National Licensing Medical Examination: a cross-sectional study.

PURPOSE: We aimed to describe the performance and evaluate the educational value of justifications provided by artificial intelligence chatbots, including GPT-3.5, GPT-4, Bard, Claude, and Bing, on the Peruvian National Medical Licensing Examination (P-NLME). METHODS: This was a cross-sectional analytical study. On July 25, 2023, each multiple-choice question (MCQ) from the P-NLME was entered into each chatbot (GPT-3, GPT-4, Bing, Bard, and Claude) 3 times. Then, 4 medical educators categorized the MCQs in terms of medical area, item type, and whether the MCQ required Peru-specific knowledge. They assessed the educational value of the justifications from the 2 top performers (GPT-4 and Bing). RESULTS: GPT-4 scored 86.7% and Bing scored 82.2%, followed by Bard and Claude, and the historical performance of Peruvian examinees was 55%. Among the factors associated with correct answers, only MCQs that required Peru-specific knowledge had lower odds (odds ratio, 0.23; 95% confidence interval, 0.09-0.61), whereas the remaining factors showed no associations. In assessing the educational value of justifications provided by GPT-4 and Bing, neither showed any significant differences in certainty, usefulness, or potential use in the classroom. CONCLUSION: Among chatbots, GPT-4 and Bing were the top performers, with Bing performing better at Peru-specific MCQs. Moreover, the educational value of justifications provided by the GPT-4 and Bing could be deemed appropriate. However, it is essential to start addressing the educational value of these chatbots, rather than merely their performance on examinations.

Intervención educativa en una enfermedad desatendida: hidatidosis/ equinococosis quística en escolares del norte de Chile / Educational intervention in a neglected disease: hydatidosis/cystic echinococcosis in schoolchildren in northern Chile

INTRODUCCIÓN: La hidatidosis es una zoonosis cuyo control más eficiente y eficaz es la promoción en salud, especialmente en la población infante-juvenil. OBJETIVO: Medir el impacto de un programa de intervención educativa sobre hidatidosis en escolares del norte de Chile. MÉTODO: Estudio intervencional educativo con fases pre y post test en 3.145 estudiantes de educación preescolar, primaria y secundaria de tres comunas. Se definieron dimensiones, unidades educativas y sistemas de evaluación incorporados en las bases curriculares. Se calculó el puntaje de logro de respuestas correctas utilizando la prueba t. Se estimó el tamaño del efecto mediante d de Cohen y análisis de varianza mixto. RESULTADOS: Se encontraron diferencias estadísticamente significativas en la muestra total y en las variables género, comuna, procedencia rural-urbana y nivel de educación. Los puntajes post intervención fueron mayores y con tamaños de efecto grandes a excepción de los primeros niveles de educación. Se hallaron puntajes mayores post test y efectos de interacción con diferencias significativas en rural-urbano y en comuna anteriormente intervenida. La dimensión mejor evaluada fue "medidas de autocuidado" y la más baja "tenencia responsable de perros". CONCLUSIONES: El programa educativo fue efectivo al ser dirigido a grupos etarios, mediante métodos pedagógicos en las bases curriculares de los establecimientos educacionales.

BACKGROUND: Hydatidosis is a zoonosis whose most efficient and effective control is health promotion, especially in children and youth. AIM: To measure the impact of an educational intervention program on hydatidosis in schoolchildren in northern Chile. METHOD: Educational interventional study was performed with pre-post test phases in 3,145 students of preschool, primary and secondary education from three districts. Dimensions, educational units and evaluation systems were defined and incorporated into the curricular bases. The correct answer achievement score was calculated using the T-test. Effect size was estimated using Cohen's d and mixed analysis of variance. RESULTS: Statistically significant differences were found in the total sample and in variables such as gender, district, rural-urban origin and level of education. Post-intervention scores were higher than pre-intervention and with large effect sizes, except for the first levels of education. Higher post-test scores and interaction effects were found with significant differences in rural-urban and in a previously intervened district. The best evaluated dimension was "self-care measures" and the lowest was "responsibly keeping dogs". CONCLUSIONS: The educational program was effective when directed to age groups, through pedagogical methods in the curricular bases of educational facilities.

Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence From a National Test.

Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests' outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees' responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options' processing are discussed, and implications for test development are considered.

[Academic satisfaction in residents of the north zone, Mexico City]. / Satisfacción académica en residentes de la zona norte, ciudad de México.

Background: Resident academic satisfaction has been linked to academic performance and quality of care. Objective: To analyze the perception of academic satisfaction in medical residents of the North Decentralized Administrative Operation Body 2 of the Instituto Mexicano del Seguro Social (IMSS), north of Mexico City. Material and methods: Cross-sectional analytical study. A total of 346 resident physicians of different specialties and academic degrees were found, enrolled in the residence halls in the northern zone of the IMSS, Mexico City from January to February 2022. The dependent variable was the level of academic satisfaction. Descriptive and inferential statistics and p ≤ 0.05 were produced for statistical significance. Results: The level of high satisfaction was 80.6% and the level of poor or low satisfaction was 1.7%. The dimension most affected in the negative bases was that of "Perception of teaching" in which 13% of the doctors surveyed refer to the need for some retraining of teachers. The best qualified dimension was "Perception of the role of autonomies" where 35.3% have an excellent perception of their personal work. The highest degree of satisfaction decreased during the second year. Conclusion: There is a high degree of satisfaction among resident doctors, however it is necessary to work to improve satisfaction in the remaining 20%.

Introducción: la satisfacción académica del residente se ha vinculado con el rendimiento académico y la calidad en la atención. Objetivo: analizar la percepción de la satisfacción académica en médicos residentes del Órgano de Operación Administrativa Desconcentrada 2 Norte del Instituto Mexicano del Seguro Social (IMSS), zona norte de la Ciudad de México. Material y métodos: estudio transversal analítico. Se encuestó a 346 médicos residentes de diferentes especialidades y grados académicos, inscritos en las sedes de residencia de la zona norte del IMSS, en la Ciudad de México de enero a febrero del 2022. La variable dependiente fue el nivel de satisfacción académica. Se utilizó estadística descriptiva, inferencial y p ≤ 0.05 para la significancia estadística. Resultados: el nivel de satisfacción alta fue del 80.6% y el nivel de satisfacción pobre o bajo fue del 1.7%. La dimensión más afectada en las puntuaciones negativas fue la de Percepción de la enseñanza, en la que el 13% de los médicos encuestados refieren la necesidad de algún reentrenamiento en los profesores. La dimensión mejor calificada fue la de Percepción del rol de autonomías, donde el 35.3% tienen una percepción excelente de su trabajo personal. El mayor grado de satisfacción se observó durante el segundo año. Conclusión: existe un alto grado de satisfacción entre médicos residentes; sin embargo, es necesario trabajar para mejorar la satisfacción en el 20% restante.

Remote vs. face-to-face activities in the teaching of renal pathophysiology in the context of social isolation during the early phase of the COVID-19 pandemic.

The advent of the COVID-19 pandemic forced medical schools around the world to adopt emergency remote learning as a resort to avoid interruption of courses. However, the effectiveness of online classes as an educational strategy has been questioned by medical educators and students. In a prospective observational study design, students enrolled in a renal physiology and pathophysiology course were exposed to either face-to-face or remote synchronous classes. Students taught online obtained significantly higher mean scores than the group who had in-person classes, both groups assessed with identical exams. Appropriate screening tests suggested that fraud is unlikely to have significantly influenced these results and that the observed differences in performance reflected increased learning by the remote group. These observations suggest that online classes can help to maintain the continuity of physiology and pathophysiology courses during periods of social isolation and may contribute to improving learning under normal conditions.NEW & NOTEWORTHY In this study, we were able to make a rare direct comparison of face-to-face and remote strategies for the teaching of undergraduate medical students in a specific area, namely, renal pathophysiology. Unexpectedly, students who attended the remote course had significantly higher grades than those who had mostly in-person classes.

Desarrollo de la Competencia Intercultural desde la formación de pregrado en la educación médica superior

La necesidad de desarrollar competencias interculturales no solo es un imperativo ético, en términos del respeto y comprensión de aquellas culturas diferentes, sino también un requisito técnico. Un profesional que pueda vincularse exitosamente con personas de diversas culturas probablemente logrará una interacción efectiva con usuarios y comunidades, un mayor nivel de confianza, que se traducirá en una mayor satisfacción con el cuidado y mejor adherencia terapéutica. En el presente artículo se analiza desde el punto de vista teórico la incorporación de la competencia intercultural en los currículos de las carreras de la salud. Se realizó una revisión bibliográfica de artículos y documentos relacionados con la competencia intercultural en la educación superior en bases de datos Pubmed y Scielo, y en páginas web de organizaciones ligadas con la temática de enseñanza y aprendizaje en educación superior, cultura y diversidad.

The need to develop intercultural competences is not only an ethical imperative, in terms of respect and understanding of those different cultures, but also a technical requirement. A professional who can successfully connect with people from diverse cultures will probably achieve effective interaction with users and communities, a higher level of trust, which will translate into greater satisfaction with care and better therapeutic adherence. This article analyzes from a theoretical point of view the incorporation of intercultural competence in the curricula of health careers. A bibliographic review of articles and documents related to intercultural competence in higher education was carried out in Pubmed and Scielo databases, and in web pages of organizations linked to the subject of teaching and learning in higher education, culture and diversity.

Diagnóstico de calidad de los resúmenes de tesis de maestría en enfermería / Quality diagnosis in abstracts of master's thesis in nursing

Resumen Objetivo: Analizar las dimensiones de estructura del resumen y referencias bibliográficas en la calidad de las tesis de los graduados de 2015 a 2020 de la maestría en Enfermería de la Universidad Nacional Autónoma de México. Materiales y Métodos: Investigación cuantitativa, descriptiva y analítica. La muestra no probabilística estuvo integrada por 96 tesis de nivel maestría de cinco generaciones de graduados 2015-2020. El instrumento se conforma por 30 ítems, se adaptó al contexto del posgrado y fue validado por expertos en cuanto a claridad en la redacción, congruencia con la dimensión a valorar y pertinencia. Se llevó a cabo un estudio piloto en una población similar a la del estudio (14 tesis de posgrado). en las cinco dimensiones consideradas en la escala. Se utilizó la Correlación de Spearman para establecer una relación de las dimensiones y la calidad del resumen de la tesis. Se utilizó el paquete estadístico SPSS versión 21. Resultados: Las correlaciones obtenidas son positivas en todos los casos. La dimensión contenido y organización presenta una mayor correlación con las de resultados y metodología y la más fuerte con las referencias bibliográficas respecto a las otras dimensiones. La dimensión metodología obtuvo mayor significancia con resultados y discusión. La dimensión de resultados con la de metodología y la de contenido y organización. La dimensión de discusión con las dimensiones de resultados y metodología. Conclusiones: El diagnóstico permitió identificar que el resumen es fundamental para visibilizar la producción de tesis de la Maestría en Enfermería en bases de datos científicas. La estructura clara y homologada de los resúmenes y de las referencias bibliográficas es un criterio importante en la autoevaluación del programa.

Abstract Objective: Analyze the structure dimensions of the summary and bibliographic references in the quality of the theses of graduates from 2015 to 2020 of the master's degree in Nursing from the National Autonomous University of Mexico. Materials and methods: Quantitative, descriptive and analytical research. The non-probabilistic sample consisted of 96 master's level theses from five generations of graduates 2015-2020. The instrument is made up of 30 items, it was adapted to the postgraduate context and was validated by experts in terms of clarity in the wording, congruence with the dimension to be assessed and relevance. A pilot study was carried out in a population similar to that of the study (14 postgraduate theses). in the five dimensions considered in the scale. Spearman's Correlation was used to establish a relationship between the dimensions and the quality of the thesis summary. The statistical package SPSS version 21 was used. Results: The correlations obtained are positive in all cases. The content and organization dimension presents a greater correlation with those of results and methodology and the strongest with the bibliographical references with respect to the other dimensions. The methodology dimension obtained greater significance with results and discussion. The dimension of results with that of methodology and that of content and organization. The discussion dimension with the dimensions of results and methodology. Conclusions: The diagnosis made it possible to identify that the summary is essential to make visible the production of the Master's theses in Nursing in scientific databases. The clear and approved structure of the abstracts and bibliographical references is an important criterion in the self-evaluation of the program.

Resultados del Mini Clinical Evaluation Exercise aplicado en la carrera de Enfermería. Universidad Estatal del Sur de Manabí, 2021

Fundamento en el contexto de las especialidades y carreras del campo de la salud se requiere implementar instrumentos de evaluación de prácticas clínicas basadas en competencias, los cuales cumplan con los criterios requeridos en correspondencia con su función. Objetivo evaluar los resultados del Mini Clinical Evaluation Exercise aplicado en la evaluación de prácticas clínicas basadas en competencias para la carrera de Enfermería. Métodos estudio comparativo con intervención educativa, en el cual se aplicó el Mini Clinical Evaluation Exercise como instrumento de evaluación de las competencias en las prácticas preprofesionales de estudiantes de la carrera de Enfermería, Facultad Ciencias de Salud de la Universidad Estatal del Sur de Manabí, en 2021. El test de Wilcoxon se usó para comparar las puntuaciones. Resultados las variables que mostraron diferencias significativas entre ambos momentos de aplicación del instrumento fueron las dimensiones: actitudinales hacia pacientes y equipo de salud (p=0,007) y valoración (p=0,10). Se observaron puntuaciones estadísticamente superiores en el segundo momento de evaluación (p=0,023). La evaluación realizada por los docentes evidenció la mejora del proceso de atención de enfermería, principalmente en cuanto a garantizar la privacidad del paciente (media de 7,46). Conclusiones el Mini Clinical Evaluation Exercise es aplicable al contexto de la educación superior de Salud en Ecuador. Se constata como una alternativa para evaluar competencias en la carrera de Enfermería.

Background in the context of specialties and careers in the health field, it is necessary to implement clinical practice evaluation instruments based on competencies, which meet the required criteria in accordance with their function. Objective to evaluate the results of the Mini Clinical Evaluation Exercise applied in the evaluation of clinical practices based on competencies for the Nursing career. Methods comparative study with educational intervention, in which the Mini Clinical Evaluation Exercise was applied as an instrument for evaluating competencies in pre-professional practices of Nursing students, Health Sciences Faculty of the Southern State University of Manabí, in 2021. The Wilcoxon test was used to compare scores. Results the variables that showed significant differences between the instrument application two moments were the dimensions: attitudes towards patients and the health team (p=0.007) and assessment (p=0.10). Statistically higher scores were observed at the second evaluation moment (p=0.023). The evaluation carried out by the teachers evidenced the improvement of the nursing care process, mainly in terms of guaranteeing the patient's privacy (mean of 7.46). Conclusions the Mini Clinical Evaluation Exercise is applicable to the context of higher health education in Ecuador. It is verified as an alternative to evaluate competencies in the Nursing career.

Rúbricas como instrumento de evaluación de ateneos en una residencia de Bioquímica Clínica de un hospital universitario. Estudio sobre su confiabilidad e impacto educativo / Rubrics as an evaluation instrument for conferences in a Clinical Biochemistry residency at a university hospital. Study on its reliability and educational impact / Rubricas como instrumento de avaliação de congressos em uma residência de Bioquímica Clínica de um hospital universitário. Estudo sobre sua confiabilidade e impacto educacional

Resumen El objetivo del estudio fue describir el uso de rúbricas para la evaluación formativa de la participación de profesionales en formación en el desarrollo de ateneos. Se realizó un estudio observacional, descriptivo, transversal y retrospectivo sobre la rúbrica de ateneos de programas de formación de posgrado de Bioquímica Clínica y se analizó la confiabilidad (por medio del coeficiente alfa de Cronbach) y su impacto educativo. Se analizaron 51 rúbricas sobre 41 ateneos de 9 profesionales en formación (residentes y becarios) por 19 evaluadores. La estimación de confiabilidad arrojó un alfa de Cronbach de 0,71 (nivel de confianza del 95%, p≤0,05). La rúbrica aquí analizada ha servido para clarificar las metas de aprendizajes de una actividad tradicional en la formación de posgrado en servicio. Se ha logrado generar un instrumento de evaluación confiable que puede ser utilizado en otras profesiones y especialidades del equipo de salud.

Abstract The objective of the study was to describe the use of rubrics for the formative evaluation of the participation of professionals in training in the development of conferences. An observational, descriptive, cross-sectional and retrospective study was carried out on the rubric of athenaeums of postgraduate training programmes in Clinical Biochemistry, analysing its reliability (by means of Cronbach's alpha coefficient) and its educational impact. Fifty-one rubrics difewere analysed on 41 athenaeums of 9 professionals in training (residents and fellows) by 19 evaluators. The reliability estimate yielded a Cronbach's alpha of 0.71 (95% confidence level, p≤0.05). The rubric analysed here has served to clarify the learning goals of a traditional activity in postgraduate training in service. It has been possible to generate a reliable evaluation instrument that can be used in other professions and specialties of the health team.

Resumo O objetivo do estudo foi descrever a utilização de rubricas para avaliação formativa da participação de profissionais em formação no desenvolvimento de ateneus. Foi realizado um estudo observacional, descritivo, transversal e retrospectivo da rubrica de ateneus dos programas de formação pós-graduação em Bioquímica Clínica, analisando a sua fiabilidade (através do coeficiente alpha de Cronbach) e o seu impacto educacional. Cinquenta e uma rubricas de 41 ateneus de 9 profissionais em formação (residentes e bolsistas) foram analisadas por 19 avaliadores. A estimativa de confiabilidade resultou em um alfa de Cronbach de 0,71 (nível de confiança de 95%, p≤0,05). A rubrica aqui analisada serviu para esclarecer os objetivos de aprendizagens de uma atividade tradicional na formação pós-graduação em serviço. Foi possível gerar um instrumento de avaliação confiável que pode ser utilizado em outras profissões e especialidades da equipe de saúde.