Time of clerkship rotations' interruption during COVID-19 and differences on Progress Test's scores.

OBJECTIVES: The transition from face-to-face to remote teaching is yet to be fully understood. In clinical training, traditional teaching must prevail because it is essential for the acquisition of skills and professionalism. However, the responses of each school to the pandemic and the decision on when to resume clerkship rotations were mixed. In this study, we aimed to analyze whether the time to resume clerkship rotations was associated with the performance of the students by using a multi-institutional Progress Test. METHODS: This is a cross-sectional study conducted at nine different Brazilian medical schools that administer the same annual Progress Test for all students. We included information from 1,470 clerkship medical students and analyzed the time of clinical training interruption as the independent variable and the student's scores as the dependent variable. RESULTS: The comparisons of the students' scores between the schools showed that there are differences; however, they cannot be attributed to the time the clerkship rotations were paused. The correlation between the schools' average scores and the time to resume clerkship rotations was not significant for the fifth year (r= -0.298, p=0.436) and for the sixth year (r= -0.440, p=0.240). By using a cubic regression model, the time to resume clerkship rotations could explain 3.4% of the 5-year students' scores (p<0.001) and 0.9% of the 6-year students, without statistical difference (p=0.085). CONCLUSIONS: The differences between the students' scores cannot be attributed to the time when the schools paused the clerkship rotations.

Time of clerkship rotations' interruption during COVID-19 and differences on Progress Test's scores

SUMMARY OBJECTIVES: The transition from face-to-face to remote teaching is yet to be fully understood. In clinical training, traditional teaching must prevail because it is essential for the acquisition of skills and professionalism. However, the responses of each school to the pandemic and the decision on when to resume clerkship rotations were mixed. In this study, we aimed to analyze whether the time to resume clerkship rotations was associated with the performance of the students by using a multi-institutional Progress Test. METHODS: This is a cross-sectional study conducted at nine different Brazilian medical schools that administer the same annual Progress Test for all students. We included information from 1,470 clerkship medical students and analyzed the time of clinical training interruption as the independent variable and the student's scores as the dependent variable. RESULTS: The comparisons of the students' scores between the schools showed that there are differences; however, they cannot be attributed to the time the clerkship rotations were paused. The correlation between the schools' average scores and the time to resume clerkship rotations was not significant for the fifth year (r= -0.298, p=0.436) and for the sixth year (r= -0.440, p=0.240). By using a cubic regression model, the time to resume clerkship rotations could explain 3.4% of the 5-year students' scores (p<0.001) and 0.9% of the 6-year students, without statistical difference (p=0.085). CONCLUSIONS: The differences between the students' scores cannot be attributed to the time when the schools paused the clerkship rotations.

Relationships between Bloom's taxonomy, judges' estimation of item difficulty and psychometric properties of items from a progress test: a prospective observational study.

BACKGROUND: Progress tests are longitudinal assessments of students' knowledge based on successive tests. Calibration of the test difficulty is challenging, especially because of the tendency of item-writers to overestimate students' performance. The relationships between the levels of Bloom's taxonomy, the ability of test judges to predict the difficulty of test items and the real psychometric properties of test items have been insufficiently studied. OBJECTIVE: To investigate the psychometric properties of items according to their classification in Bloom's taxonomy and judges' estimates, through an adaptation of the Angoff method. DESIGN AND SETTING: Prospective observational study using secondary data from students' performance in a progress test applied to ten medical schools, mainly in the state of São Paulo, Brazil. METHODS: We compared the expected and real difficulty of items used in a progress test. The items were classified according to Bloom's taxonomy. Psychometric properties were assessed based on their taxonomy and fields of knowledge. RESULTS: There was a 54% match between the panel of experts' expectations and the real difficulty of items. Items that were expected to be easy had mean difficulty that was significantly lower than that of items that were expected to be medium (P < 0.05) or difficult (P < 0.01). Items with high-level taxonomy had higher discrimination indices than low-level items (P = 0.026). We did not find any significant differences between the fields in terms of difficulty and discrimination. CONCLUSIONS: Our study demonstrated that items with high-level taxonomy performed better in discrimination indices and that a panel of experts may develop coherent reasoning regarding the difficulty of items.

Relationships between Bloom's taxonomy, judges' estimation of item difficulty and psychometric properties of items from a progress test: a prospective observational study

ABSTRACT BACKGROUND: Progress tests are longitudinal assessments of students' knowledge based on successive tests. Calibration of the test difficulty is challenging, especially because of the tendency of item-writers to overestimate students' performance. The relationships between the levels of Bloom's taxonomy, the ability of test judges to predict the difficulty of test items and the real psychometric properties of test items have been insufficiently studied. OBJECTIVE: To investigate the psychometric properties of items according to their classification in Bloom's taxonomy and judges' estimates, through an adaptation of the Angoff method. DESIGN AND SETTING: Prospective observational study using secondary data from students' performance in a progress test applied to ten medical schools, mainly in the state of São Paulo, Brazil. METHODS: We compared the expected and real difficulty of items used in a progress test. The items were classified according to Bloom's taxonomy. Psychometric properties were assessed based on their taxonomy and fields of knowledge. RESULTS: There was a 54% match between the panel of experts' expectations and the real difficulty of items. Items that were expected to be easy had mean difficulty that was significantly lower than that of items that were expected to be medium (P < 0.05) or difficult (P < 0.01). Items with high-level taxonomy had higher discrimination indices than low-level items (P = 0.026). We did not find any significant differences between the fields in terms of difficulty and discrimination. CONCLUSIONS: Our study demonstrated that items with high-level taxonomy performed better in discrimination indices and that a panel of experts may develop coherent reasoning regarding the difficulty of items.

Teste de Progresso: uma Ferramenta Avaliativa para a Gestão Acadêmica / Progress Test: an Evaluative Tool for Academic Management

Este estudo teve por objetivo averiguar as potencialidades do Teste de Progresso (TP) no auxílio à gestão acadêmica. Foram analisados os desempenhos dos estudantes do curso de Medicina no TP aplicado em 2008 e reaplicado sem modificações em 2011. Os resultados demonstraram que, no TP de 2008, a progressão ocorreu a cada dois anos de curso, e no TP 2011 esta progressão iniciou-se somente a partir da terceira série. Nas áreas de Saúde Coletiva, Clínica Cirúrgica e Áreas Básicas, não houve acúmulo de conhecimentos gradativo desde o início do curso, o que merece reflexões por parte dos gestores do currículo.

This study aimed to investigate the potential of progress testing to help academic management. To this end, the performances of medical students in the Progress Test applied in 2008 and reapplied without modifications in 2011 were analysed. The results showed that in the Progress Test of 2008 progression occurred once every two years of the course and in TP 2011 progression started only from the third year onwards. In the areas of Public Health, Surgery and Basic Sciences there was no gradual accumulation of knowledge since the beginning of the course, which merits reflection on behalf of the curriculum managers.

Diagnóstico de anemia em crianças de 0 a 3 anos: uma necessidade de recomendações / Diagnosis of Anemia in Children Aged 0-3 - A Need for Recommendations

Este estudo tem a finalidade de quantificar os diagnósticos de anemia em crianças de 0 a 3 anos de idade, realizados pela Unidade de Saúde da Família Vila Real, localizada em Marília/SP, e avaliar a necessidade de implementação de recomendações para diagnóstico da anemia. O estudo é retrospectivo e abrangeu amostra de crianças de 0 a 3 anos de idade atendidas nessa unidade de saúde, sendo os dados dos hemogramas e dos prontuários coletados, computados e analisados no programa Epi-Info. Entre as 278 crianças atendidas pelo posto, 63 (22,7 por cento) tinham hemograma no prontuário, sem uniformidade nas idades na época do exame. Dessas, 49 (77,8 por cento) tinham hemoglobina<11,0 g/dl; 41 (65,07 por cento), hematócrito abaixo do valor limite; e 38 (61,3 por cento), volume corpuscular médio reduzido. Essas são evidências de que existe alta prevalência de anemia e deficiência de ferro na área estudada, com forte indicativo da importância de mais projetos e ações básicas, visando à diminuição do número de crianças com anemia. Observaram-se crianças com diferentes idades na época do exame, demonstrando a necessidade de recomendações para seu diagnóstico.