Predicting students' academic progress and related attributes in first-year medical students: an analysis with artificial neural networks and Naïve Bayes.

BACKGROUND: Dropout and poor academic performance are persistent problems in medical schools in emerging economies. Identifying at-risk students early and knowing the factors that contribute to their success would be useful for designing educational interventions. Educational Data Mining (EDM) methods can identify students at risk of poor academic progress and dropping out. The main goal of this study was to use machine learning models, Artificial Neural Networks (ANN) and Naïve Bayes (NB), to identify first year medical students that succeed academically, using sociodemographic data and academic history. METHODS: Data from seven cohorts (2011 to 2017) of admitted medical students to the National Autonomous University of Mexico (UNAM) Faculty of Medicine in Mexico City were analysed. Data from 7,976 students (2011 to 2017 cohorts) of the program were included. Information from admission diagnostic exam results, academic history, sociodemographic characteristics and family environment was used. The main dataset included 48 variables. The study followed the general knowledge discovery process: pre-processing, data analysis, and validation. Artificial Neural Networks (ANN) and Naïve Bayes (NB) models were used for data mining analysis. RESULTS: ANNs models had slightly better performance in accuracy, sensitivity, and specificity. Both models had better sensitivity when classifying regular students and better specificity when classifying irregular students. Of the 25 variables with highest predictive value in the Naïve Bayes model, percentage of correct answers in the diagnostic exam was the best variable. CONCLUSIONS: Both ANN and Naïve Bayes methods can be useful for predicting medical students' academic achievement in an undergraduate program, based on information of their prior knowledge and socio-demographic factors. Although ANN offered slightly superior results, Naïve Bayes made it possible to obtain an in-depth analysis of how the different variables influenced the model. The use of educational data mining techniques and machine learning classification techniques have potential in medical education.

Bridging Validity Frameworks in Assessment: Beyond Traditional Approaches in Health Professions Education.

Construct: High-stakes assessments measure several constructs, such as knowledge, competencies, and skills. In this case, validity evidence for test scores' uses and interpretations is of utmost importance, because of the consequences for everyone involved in their development and implementation. Background: Educational assessment requires an appropriate understanding and use of validity frameworks; however, health professions educators still struggle with the conceptual challenges of validity, and frequently validity analyses have a narrow focus. Important obstacles are the plurality of validity frameworks and the difficulty of grounding these abstract concepts in practice. Approach: We reviewed the validity frameworks literature to identify the main elements of frequently used models (Messick and Kane's) and proposed linking frameworks including Russell's recent overarching proposal. Examples are provided with commonly used assessment instruments in health professions education. Findings: Several elements in these frameworks can be integrated into a common approach, matching and aligning Messick's sources of validity with Kane's four inference types. Conclusions: This proposal to contribute evidence for assessment inferences may provide guidance to understanding the use of validity evidence in applied settings. The evolving field of validity research provides opportunities for its integration and practical use in health professions education.

Health sciences undergraduate students academic track records.

BACKGROUND: Academic track record analysis is essential for evaluating the training of students and the structure of higher education study programs, which allows diagnosing and preventing educational lag and school dropout. OBJECTIVE: To analyze the differences in academic track records of UNAM health sciences undergraduate students from generations 2001 to 2016. MATERIAL AND METHODS: Study of real cohorts; graduation and lag rates were calculated. ANOVA was used to contrast the graduation rates between campuses by undergraduate program and time. To identify critical periods, survival functions were used with Kaplan-Meier's method. RESULTS: The lowest percentages of lag were observed in nursing and medicine students; nursing students had the highest graduation rates, especially at Zaragoza campus; dentistry students had the lowest graduation rates and the highest dropout and lag rates. Women showed higher graduation rates and lower risk of dropout and lag. CONCLUSIONS: Nursing, medicine and psychology undergraduate students at Zaragoza and Iztacala campuses, with modular programs, achieved the highest graduation percentages and the lowest dropout and lag rates.

ANTECEDENTES: El análisis de las trayectorias académicas es fundamental para evaluar la formación de los estudiantes y la estructura de los programas de estudio de educación superior, lo que permite diagnosticar y prevenir el rezago y abandono escolar. OBJETIVO: Analizar las diferencias en las trayectorias académicas de los estudiantes de las licenciaturas en ciencias de la salud de la UNAM de las generaciones 2001 a 2016. MATERIAL Y MÉTODOS: Estudio de cohortes reales; se calcularon tasas de egreso y rezago. Se realizó ANOVA para contrastar el egreso entre planteles por carrera y tiempo. Para identificar los períodos críticos se utilizaron funciones de supervivencia con el método de Kaplan-Meier. RESULTADOS: En las licenciaturas en enfermería y medicina se observaron los menores porcentajes de rezago; enfermería presentó los mayores porcentajes de egreso, sobre todo en la Facultad de Estudios Superiores Zaragoza; odontología mostró los menores índices de egreso y mayores índices de abandono y rezago. Las mujeres mostraron mayor egreso y menor riesgo de abandono y rezago. CONCLUSIONES: Los estudiantes de las licenciaturas en enfermería, medicina y psicología de las facultades de estudios superiores Zaragoza e Iztacala, con programas modulares, alcanzaron los mayores porcentajes de egreso y menores índices de abandono y rezago.

Medicine and the metaverse: current applications and future.

Innovative technologies such as the metaverse and chat GPT-4 (based on artificial intelligence) are present in the daily discourse of society; recently, they have been introduced into medical practice and are bringing about important changes. In the case of the metaverse ("beyond the universe"), various medical schools and departments around the world are beginning to use it as an innovative strategy for teaching subjects such as anatomy, histology, ophthalmology, and simulation in parallel (virtual) worlds for learning and supervision of surgeries, as well as for other applications in medical education and in the doctor-patient relationship. Although it should be regarded as an area of opportunity for the transformation of medicine, it is important to consider the various limitations and risks of the metaverse in medical practice, student training, and physicians' relationship with the health problems they have to deal with in their practice.

Las innovadoras tecnologías del metaverso y el chat GPT4 (basado en inteligencia artificial) están presentes en el discurso cotidiano de la sociedad; recientemente se han introducido en la práctica médica y están provocando importantes cambios. En cuanto al metaverso ("después del universo"), diversas escuelas y facultades de medicina del mundo comienzan a utilizarlo como una estrategia innovadora dirigida a la enseñanza de materias como anatomía, histología, oftalmología y simulación en mundos paralelos (virtuales) para el aprendizaje y supervisión de cirugías, así como para otras aplicaciones en educación médica y en la relación médico-paciente. Si bien debe tomarse en cuenta como un área de oportunidad para la transformación de la medicina, es importante considerar las diversas limitaciones y riesgos del metaverso en la práctica médica, la formación de estudiantes y la relación del médico con los problemas de salud a los que se enfrenta en su práctica.

ChatGPT: opportunities and risks in the fields of medical care, teaching, and research.

ChatGPT is a virtual assistant with artificial intelligence (AI) that uses natural language to communicate, i.e., it holds conversations as those that would take place with another human being. It can be applied at all educational levels, including medical education, where it can impact medical training, research, the writing of scientific articles, clinical care, and personalized medicine. It can modify interactions between physicians and patients and thus improve the standards of healthcare quality and safety, for example, by suggesting preventive measures in a patient that sometimes are not considered by the physician for multiple reasons. ChatGPT potential uses in medical education, as a tool to support the writing of scientific articles, as a medical care assistant for patients and doctors for a more personalized medical approach, are some of the applications discussed in this article. Ethical aspects, originality, inappropriate or incorrect content, incorrect citations, cybersecurity, hallucinations, and plagiarism are some examples of situations to be considered when using AI-based tools in medicine.

ChatGPT es un asistente virtual con inteligencia artificial que utiliza lenguaje natural para comunicarse, es decir, mantiene conversaciones como las que se tendrían con otro humano. Puede aplicarse en educación a todos los niveles, que incluye la educación médica, en donde puede impactar en la formación, la investigación, la escritura de artículos científicos, la atención clínica y la medicina personalizada. Puede modificar la interacción entre médicos y pacientes para mejorar los estándares de calidad de la atención médica y la seguridad, por ejemplo, al sugerir medidas preventivas en un paciente que en ocasiones no son consideradas por el médico por múltiples causas. Los usos potenciales del ChatGPT en la educación médica, como una herramienta de ayuda en la redacción de artículos científicos, un asistente en la atención para pacientes y médicos para una práctica más personalizada, son algunas de las aplicaciones que se analizan en este artículo. Los aspectos éticos, originalidad, contenido inapropiado o incorrecto, citas incorrectas, ciberseguridad, alucinaciones y plagio son ejemplos de las situaciones a tomar en cuenta al usar las herramientas basadas en inteligencia artificial en medicina.

Analysis of physical learning spaces in a university hospital: A case study.

BACKGROUND: Attributes of physical learning spaces can facilitate or hinder learning. There are few studies about this topic in hospitals. The objective of this study was to explore the characteristics of physical learning spaces in a university hospital. METHODS: The setting was a large research-oriented public university hospital in Mexico City, affiliated with the National Autonomous University of Mexico. An intrinsic case study design was conducted with two instruments: a questionnaire to identify physical learning spaces and their attributes; the Learning Space Rating System (LSRS), an instrument used to evaluate spaces' characteristics that promote effective learning. RESULTS: 49 medical students and 60 internal medicine residents responded to the questionnaire. The attributes with the highest importance for students were: instructor physical availability, silence, comfort of the seating furniture, and Internet access. The study authors directly performed an evaluation of the learning spaces using the LSRS tool, the sections' scores were: clinical ward discussion room (74%), external consultation unit (71%), auditorium (70%), classroom (68%), library (66%), and the hospitalization room (61%). CONCLUSIONS: Physical learning spaces in medical training can be formally assessed to identify the attributes that students consider relevant for learning and provide needed information for redesign and reuse of spaces. Medical education scholars and trainees should be involved in the design and evaluation of university and hospital buildings.

Antibiotic prescription errors: the relationship with clinical competence in junior medical residents.

BACKGROUND: A large portion of prescribing errors can be attributed to deficiencies in medication knowledge. These errors are preventable and most often occur at the time of prescription. Antimicrobials are the drug class most common incorrectly prescribed. OBJECTIVE: To characterize the relationship between clinical competence and antibiotic prescription errors. We also investigated the frequency and severity of antibiotic prescription errors to identify items and attributes of clinical competence which are correlated with the antibiotic prescription error ratio. METHOD: A cross-sectional study was applied to assess clinical competence of junior medical residents in two reference academic hospitals and a regional hospital in Mexico City. It was conducted during February 2019. We used an infectious disease Objective Structured Clinical Examination (OSCE) to assess clinical competence and a measure of frequency, and severity of antibiotic prescription errors. RESULTS: The number of eligible participants was ~ 255 (hospital meeting attendance), and the number of residents in this study were 51 (~ 20%), 31 were female (60.8%). The mean OSCE score was 0.692 ± 0.073. The inter-item (Cronbach's alpha = 0.927) and inter-station internal consistency was adequate (Cronbach's alpha = 0.774). The G coefficient in generalizability theory analysis was 0.84. The antibiotic prescription error ratio was 45.1% ± 7%. The most frequent category of severity of antibiotic prescription errors was category E (errors that may contribute to or result in temporary harm to the patient and require intervention), 235 (65.2%). We observed a negative and significant correlation between clinical competence and antibiotic prescription errors (r = -0.33, p < 0.05, CI95% -0.57 to -0.07), which remained significant after controlling for the effect of gender and time since graduation from medical school (r = -0.39, p < 0.01, CI95% -0.625 to -0.118). Using exploratory factor analysis we identified two factors, which explained 69% of the variance in clinical competence, factor 1 evaluated socio-clinical skills and factor 2 evaluated diagnostic-therapeutic skills. Factor 2 was correlated with antibiotic prescription error ratio (r = -0.536, p < 0.001). CONCLUSIONS: We observed a negative correlation between clinical competence and antibiotic prescription error ratio in graduated physicians who have been accepted in a medical specialty. The therapeutic plan, which is a component of the clinical competence score, and the prescription skills had a negative correlation with antibiotic prescription errors. The most frequent errors in antibiotic prescriptions would require a second intervention.

Pros and cons of innovations in medical education.

Medical education has implemented various innovative strategies with the purpose to attain better learning achievements. An evaluation is made of the experiences in the competencies approach, new learning technologies, curricular alternatives, professional evaluation and distance education technologies in order to locate them in the areas they belong.

La educación médica ha puesto en práctica diversas estrategias innovadoras con el propósito de alcanzar mejores logros de aprendizaje. Se hace una evaluación de las experiencias relacionadas con el enfoque por competencias, las nuevas tecnologías educativas, las alternativas curriculares, la profesionalización de la evaluación y las técnicas educativas a distancia, para ubicarlas en el lugar que les corresponde.

Specialty choice determinants among Mexican medical students: a cross-sectional study.

BACKGROUND: The choice of medical specialty is related to multiple factors, students' values, and specialty perceptions. Research in this area is needed in low- and middle-income countries, where the alignment of specialty training with national healthcare needs has a complex local interdependency. The study aimed to identify factors that influence specialty choice among medical students. METHODS: Senior students at the National Autonomous University of Mexico (UNAM) Faculty of Medicine answered a questionnaire covering demographics, personal experiences, vocational features, and other factors related to specialty choice. Chi-square tests and factor analyses were performed. RESULTS: The questionnaire was applied to 714 fifth-year students, and 697 provided complete responses (response rate 81%). The instrument Cronbach's alpha was 0.8. The mean age was 24 ± 1 years; 65% were women. Eighty percent of the students wanted to specialize, and 60% had participated in congresses related to the specialty of interest. Only 5% wanted to remain as general practitioners. The majority (80%) wanted to enter a core specialty: internal medicine (29%), general surgery (24%), pediatrics (11%), gynecology and obstetrics (11%) and family medicine (4%). The relevant variables for specialty choice were grouped in three dimensions: personal values that develop and change during undergraduate training, career needs to be satisfied, and perception of specialty characteristics. CONCLUSIONS: Specialty choice of medical students in a middle-income country public university is influenced by the undergraduate experience, the desire to study a subspecialty and other factors (including having skills related to the specialty and type of patients).

Educational experiences in a master of health professions education program at the National Autonomous University of Mexico: A qualitative study.

Background: Medical schools and healthcare institutions need leaders with formal training in education, in order to provide quality medical teaching. An answer to this need lies in the graduate programs of health professions education. Many programs exist, but there is a dearth of publications about their educational processes and experiences. The purpose of this study was to explore the teaching and learning experiences of students, teachers, and graduates of the Master in Health Professions Education (MHPE) program at the National Autonomous University of Mexico (UNAM). Methods: A qualitative approach was used with focus group discussions with students, graduates, and teachers, to explore their opinions, feelings, and experiences about the program. Purposeful sampling of participants was done. Focus group guides were developed for the different study groups; testimonies were codified and categorized with axial coding and a constant comparison method. Results: Testimonies from 19 participants in three focus groups were obtained (five graduates, seven current students, and seven teachers). The data were grouped in seven thematic categories: expectations, feedback of research projects, the tutorial process, teaching strategies, usefulness of what was learned, professional development, and assessment. Positive elements of the program were identified as well as areas in need of improvement. Discussion: The MHPE program at UNAM has been a positive experience for students and mostly fulfilled their expectations, they learned the basic theories and practical aspects of teaching, learning, and assessment in the health professions. Some areas need improvement, such as tutor performance and timely feedback to the students. Graduates think the competencies acquired in the program are useful for their professional practice. This information will be used to improve the program. There is a need to meet international standards in MHPE programs.

[La analítica del aprendizaje en educación médica: ¿punto de inflexión?]

La analítica del aprendizaje es una disciplina novedosa que tiene un enorme potencial para mejorar la calidad de la educación médica y la evaluación del aprendizaje. Se define como: "la medición, recopilación, análisis y reporte de datos sobre los alumnos y sus contextos, con el propósito de entender y optimizar el aprendizaje y los entornos en que ocurre". En las últimas décadas, la aparición de grandes volúmenes de datos (big data), acompañada de una rápida evolución en la minería de datos educativos, la aparición de tecnologías sofisticadas para analizar y visualizar datos de cualquier tipo, así como la disponibilidad de dispositivos móviles con conectividad permanente, mayor velocidad de procesamiento y capacidad de recuperación de información, han generado un contexto que favorece el uso de la analítica del aprendizaje en la medicina clínica y la educación médica. En este artículo se describe la historia reciente del concepto de analítica del aprendizaje, sus ventajas y desventajas en educación superior, así como sus aplicaciones en la enseñanza de las ciencias de la salud y la evaluación educativa. Es necesario que la comunidad de educadores médicos conozca la analítica del aprendizaje, para ser capaces de integrarla en su contexto eficaz y oportunamente.

Learning analytics is an innovative discipline that has an enormous potential to improve the quality of medical education and learning assessment. It is defined as: "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs". In recent decades, the appearance of large volumes of data (big data), accompanied by a quick evolution of educational data mining techniques, the emergence of sophisticated technologies to analyze and visualize any type of data, as well as the availability of permanently-connected mobile electronic devices, higher processing speed and capacity of information retrieval, have generated a context that favors the use of learning analytics in clinical medicine and medical education. In this paper, the recent history of the concept of learning analytics is described, as well as its advantages and disadvantages in higher education, and its applications in the teaching of health sciences and educational assessment. It is necessary for the community of medical educators to be acquainted with learning analytics, in order to be able to integrate it to our context in an efficacious and timely manner.

Factores relacionados con la elección de una especialidad en médicos residentes mexicanos.

OBJECTIVE: This study explored choice factors in four specialties in Mexico. METHOD: Mixed methods design. Qualitative phase: four focus groups with first-year residents, to obtain information of how specialty choice was done. With this information a web-based cross-sectional questionnaire was applied to residents registered in the 1st year of Postgraduate Studies Division, UNAM. RESULTS: 32 residents participated in qualitative phase and for the quantitative phase, the survey was answered by 35 surgical, 28 gynecology, 61 internal medicine and 62 pediatric residents. The specialty choice decision was done during the last years of the medical career. The majority of the resident choice was a subspecialty after the general residency. The type of patient was more crucial to choose pediatrics while a good academic program was for internal medicine. Negative models and bullying were decisive to rule out surgery as well as a not well-known hospital was to rule out pediatrics. CONCLUSION: The specialty choice is done during undergraduate training, with the intention of doing a subspecialty. Demographic and personality traits were identified.

[Diagnostic and formative assessment of competencies at the beginning of undergraduate medical internship]. / valuación diagnóstica y formativa de competencias en estudiantes de medicina a su ingreso al internado médico de pregrado.

INTRODUCTION: Research on diagnostic and formative assessment competencies during undergraduate medical training is scarce in Latin America. OBJECTIVE: To assess the level of clinical competence of students at the beginning of their medical internship in a new curriculum. METHODS: This was an observational cross-sectional study in UNAM Faculty of Medicine students in Mexico City: a formative assessment of the second class of Curriculum 2010 students as part of the integral evaluation of the program. The assessment had two components: theoretical and practical. RESULTS: We assessed 577 students (65.5%) of the 880 total population that finished the 9th semester of Curriculum 2010. The written exam consisted of 232 items, with a mean of 61.0 ± 19.6, a difficulty index of 0.61, and Cronbach's alpha of 0.89. The mean of the objective structured clinical examination (OSCE) was 62.2 ± 16.8, with a mean Cronbach's alpha of 0.51. Results were analyzed by knowledge area and exam stations. CONCLUSIONS: The overall results provide evidence that students achieve sufficiently the competencies established in the curriculum at the beginning of the internship, that they have the necessary foundation for learning new and more complex information, and integrate it with existing knowledge to achieve significant learning and continue their training.

[Level of clinical competence in seven cohorts of medical students at the end of the internship]. / Grado de competencia clínica de siete generaciones de estudiantes al término del internado médico de pregrado.

INTRODUCTION: Objective structured clinical examination is the instrument with more validated evidence to assess the degree of clinical competence of medical students. OBJECTIVES: To assess the degree of clinical competence of medical students at the end of their internship; to assess the reliability of the instruments with G theory. METHODS: This was an observational, longitudinal, and comparative study. The target population was composed of 5,399 interns of seven generations that finished their internship at the Faculty of Medicine of UNAM, between 2009 and 2015. The instrument used was 18 OSCE stations, three in each subject of the internship. RESULTS: The undergraduate medical interns show a sufficient degree of clinical competence to be general practitioners. The laboratory interpretation and physical examination had the highest scores. The interpretation of imaging studies was the component with the lowest score. The Family Medicine disciplinal area had the highest average score in the OSCE; in contrast, Pediatrics obtained the lowest score on average. The reliability was measured with Generalizability Theory and ranged between 0.81 and 0.93. CONCLUSIONS: The clinical competence of undergraduate medical interns is considered sufficient. The results also show the subjects that require educational interventions to improve clinical competence of students.

Eight years' experience with a Medical Education Journal Club in Mexico: a quasi-experimental one-group study.

BACKGROUND: A time-honored strategy for keeping up to date in medicine and improving critical appraisal skills is the Journal Club (JC). There are several reports of its use in medicine and allied health sciences but almost no reports of JC focused on medical education. The purpose of the study is to describe and evaluate an eight years' experience with a medical education Journal Club (MEJC). METHODS: We started a monthly medical education JC in 2006 at UNAM Faculty of Medicine in Mexico City. Its goal is to provide faculty with continuing professional development in medical education. A discussion guide and a published paper were sent 2 weeks before sessions. We reviewed the themes and publication types of the papers used in the sessions, and in June-July 2014 administered a retrospective post-then-pre evaluation questionnaire to current participants that had been regular attendees to the JC for more than 2 years. The retrospective post-then-pre comparisons were analyzed with Wilcoxon signed-rank test. Effect sizes were calculated for the pre-post comparisons with Cohen's r. RESULTS: There have been 94 MEJC sessions until July 2014. Average attendance is 20 persons, a mix of clinicians, educators, psychologists and a sociologist. The articles were published in 32 different journals, and covered several medical education themes (curriculum, faculty development, educational research methodology, learning methods, assessment, residency education). 22 Attendees answered the evaluation instrument. The MEJC had a positive evaluation from good to excellent, and there was an improvement in self-reported competencies in medical education literature critical appraisal and behaviors related to the use of evidence in educational practice, with a median effect size higher than 0.5. The evaluation instrument had a Cronbach's alpha of 0.96. CONCLUSIONS: A periodic Medical Education Journal Club can improve critical appraisal of the literature, and be maintained long-term using evidence-based strategies. This activity is a useful adjunct to the scholarship of teaching.

A typology of uncertainty derived from an analysis of critical incidents in medical residents: A mixed methods study.

BACKGROUND: Medical uncertainty is inherently related to the practice of the physician and generally affects his or her patient care, job satisfaction, continuing education, as well as the overall goals of the health care system. In this paper, some new types of uncertainty, which extend existing typologies, are identified and the contexts and strategies to deal with them are studied. METHODS: We carried out a mixed-methods study, consisting of a qualitative and a quantitative phase. For the qualitative study, 128 residents reported critical incidents in their clinical practice and described how they coped with the uncertainty in the situation. Each critical incident was analyzed and the most salient situations, 45 in total, were retained. In the quantitative phase, a distinct group of 120 medical residents indicated for each of these situations whether they have been involved in the described situations and, if so, which coping strategy they applied. The analysis examines the relation between characteristics of the situation and the coping strategies. RESULTS: From the qualitative study, a new typology of uncertainty was derived which distinguishes between technical, conceptual, communicational, systemic, and ethical uncertainty. The quantitative analysis showed that, independently of the type of uncertainty, critical incidents are most frequently resolved by consulting senior physicians (49 % overall), which underscores the importance of the hierarchical relationships in the hospital. The insights gained by this study are combined into an integrative model of uncertainty in medical residencies, which combines the type and perceived level of uncertainty, the strategies employed to deal with it, and context elements such as the actors present in the situation. The model considers the final resolution at each of three levels: the patient, the health system, and the physician's personal level. CONCLUSIONS: This study gives insight into how medical residents make decisions under different types of uncertainty, giving account of the context in which the interactions take place and of the strategies used to resolve the incidents. These insights may guide the development of organizational policies that reduce uncertainty and stress in residents during their clinical training.

Evaluation of a Biomedical Informatics course for medical students: a pre-posttest study at UNAM Faculty of Medicine in Mexico.

BACKGROUND: Biomedical Informatics (BMI) education in medical schools is developing a sound curricular base, but there are few published reports of their educational usefulness. The goal of this paper is to assess knowledge change and satisfaction in medical students after a BMI curriculum. METHODS: The National Autonomous University of México Faculty of Medicine (UNAM) recently implemented a curricular reform that includes two BMI sequential courses (BMI-1 and BMI-2). The research design was one-group pretest-posttest. An objective test with evidence of validity was used for knowledge measurement. A satisfaction questionnaire was applied at the end of the courses. Two-tailed paired Student's t-tests were applied, comparing knowledge scores in the pre and post-test for each course. RESULTS: The study included student cohorts during two consecutive academic years. The 2013 BMI-1 course (n = 986 students) knowledge pretest score was 43.0 ± 8.6 (mean percent correct ± SD), and the post-test score was 57.7 ± 10.3 (p < 0.001); the 2014 BMI-1 (n = 907) pretest score was 43.7 ± 8.5, and the post-test was 58.1 ± 10.5 (p < 0.001). The 2012 BMI-2 course (n = 683) pretest score was 26.3 ± 7.9, the post-test score was 44.3 ± 13.3 (p < 0.001); the 2013 BMI-2 (n = 926) pretest score was 27.5 ± 7.5, and the post-test was 42.0 ± 11.0 (p < 0.001). The overall opinion of the students regarding the course was from good to excellent, with a response rate higher than 90%. The satisfaction questionnaires had high reliability (Cronbach's alpha of 0.93). CONCLUSIONS: The study shows a significant increase in BMI knowledge after an educational intervention in four medical student cohorts, and an overall positive evaluation by the students. Long-term follow-up is needed, as well as controlled studies of BMI educational interventions using performance endpoints.

[Diagnostic assessment of knowledge and competencies in medical students at the end of the second year of medical school: the challenge of building an airplane while it's flying]. / Evaluación diagnóstica de conocimientos y competencias en estudiantes de medicina al término del segundo año de la carrera: el reto de construir el avión mientras vuela.

INTRODUCTION: In Latin America there is almost no published information about knowledge retention and formative assessment of competencies in medical students, during medical school training and curricular changes. OBJECTIVE: To assess knowledge level and clinical competencies in medical students at the end of the second year in a new curriculum. METHODS: Observational, cross-sectional study in UNAM Faculty of Medicine students. A diagnostic evaluation was performed in the first class of the "Plan of Studies 2010" curriculum, as part of an integral program evaluation strategy. The assessment had two components: theoretical and practical. RESULTS: 456 (87%) of the 524 students that successfully completed the second year of Plan 2010 were assessed. The written test had 211 items, a mean score of 60 ± 14.5, mean difficulty index of 0.60, reliability with Cronbach's alpha of 0.85. The OSCE mean global score was 58 ± 9, Cronbach's alpha of 0.36, and G-coefficient of 0.48, and results were provided for each station. Results by area of knowledge, course, and station were reported. CONCLUSIONS: The results in general are acceptable, compared with previous written evaluations at the end of the second year, suggesting that the new program is achieving its educational goals. Competencies were formally assessed for the first time in our Institution, establishing a starting point for follow-up. The study provided useful information to the institution, teachers and students.

[Clinical competence evaluation using the Objective Structured Clinical Examination (OSCE) in medical internship at UNAM]. / Evaluación de la competencia clínica con el examen clínico objetivo estructurado en el internado médico de la Universidad Nacional Autónoma de México.

INTRODUCTION: The Objective Structured Clinical Examination (OSCE) is a widely used measurement tool to assess clinical competence in the health sciences. There is little published evidence of its use in Mexican medical schools. OBJECTIVE: To assess clinical competence in medical students with an OSCE, before and after the Medical Internship. METHODS: Prospective cohort study, pre- post-test research design. The assessed population was medical students at UNAM Faculty of Medicine in Mexico in their Internship year. The instrument was an 18-stations OSCE, three stations per academic area of the Internship curriculum. RESULTS: We assessed the clinical competence of 278 students in a pretest OSCE when starting the Internship year, and tested them 10 months later with an equivalent post-test OSCE. The sample of students was 30.4% of the total Internship population. Test reliability with Cronbach's alpha was 0.62 in the pre-test and 0.64 in the post-test. The global mean score in the pretest OSCE was 55.6 ± 6.6 and in the post-test 63.2 ± 5.7 (p < 0.001), with a Cohen's d of 1.2. CONCLUSIONS: The clinical competence of medical students measured with an OSCE is higher after the medical internship year. This difference suggests that the internship can influence the development of clinical competence in medical students.

Teaching of evidence-based medicine to medical students in Mexico: a randomized controlled trial.

BACKGROUND: Evidence-Based Medicine (EBM) is an important competency for the healthcare professional. Experimental evidence of EBM educational interventions from rigorous research studies is limited. The main objective of this study was to assess EBM learning (knowledge, attitudes and self-reported skills) in undergraduate medical students with a randomized controlled trial. METHODS: The educational intervention was a one-semester EBM course in the 5th year of a public medical school in Mexico. The study design was an experimental parallel group randomized controlled trial for the main outcome measures in the 5th year class (M5 EBM vs. M5 non-EBM groups), and quasi-experimental with static-groups comparisons for the 4th year (M4, not yet exposed) and 6th year (M6, exposed 6 months to a year earlier) groups. EBM attitudes, knowledge and self-reported skills were measured using Taylor's questionnaire and a summative exam which comprised of a 100-item multiple-choice question (MCQ) test. RESULTS: 289 Medical students were assessed: M5 EBM=48, M5 non-EBM=47, M4=87, and M6=107. There was a higher reported use of the Cochrane Library and secondary journals in the intervention group (M5 vs. M5 non-EBM). Critical appraisal skills and attitude scores were higher in the intervention group (M5) and in the group of students exposed to EBM instruction during the previous year (M6). The knowledge level was higher after the intervention in the M5 EBM group compared to the M5 non-EBM group (p<0.001, Cohen's d=0.88 with Taylor's instrument and 3.54 with the 100-item MCQ test). M6 Students that received the intervention in the previous year had a knowledge score higher than the M4 and M5 non-EBM groups, but lower than the M5 EBM group. CONCLUSIONS: Formal medical student training in EBM produced higher scores in attitudes, knowledge and self-reported critical appraisal skills compared with a randomized control group. Data from the concurrent groups add validity evidence to the study, but rigorous follow-up needs to be done to document retention of EBM abilities.