Insights into undergraduate medical student selection tools: a systematic review and meta-analysis.

PURPOSE: Evaluating medical school selection tools is vital for evidence-based student selection. With previous reviews revealing knowledge gaps, this meta-analysis offers insights into the effectiveness of these selection tools. METHODS: A systematic review and meta-analysis were conducted applying the following criteria: peer-reviewed articles available in English, published from 2010 and which include empirical data linking performance in selection tools with assessment and dropout outcomes of undergraduate entry medical programs. Systematic reviews, meta-analyses, general opinion pieces, or commentaries were excluded. Effect sizes (ESs) of the predictability of academic and clinical performance within and by the end of the medicine program were extracted, and the pooled ESs were presented. RESULTS: Sixty-seven out of 2,212 articles were included, which yielded 236 ESs. Previous academic achievement predicted medical program academic performance (Cohen's d=0.697 in early program; 0.619 in end of program) and clinical exams (0.545 in end of program). Within aptitude tests, verbal reasoning and quantitative reasoning predicted academic achievement in the early program and in the last years (0.704 & 0.643, respectively). Overall aptitude tests predicted academic achievement in both the early and last years (0.550 & 0.371, respectively). Neither panel interviews, multiple mini-interviews, nor situational judgement tests (SJT) yielded statistically significant pooled ES. CONCLUSION: Current evidence suggests that learning outcomes are predicted by previous academic achievement and aptitude tests. The predictive value of SJT and topics such as selection algorithms, features of interview (e.g., content of the questions) and the way the interviewers' reports are used, warrant further research.

Test preparation breeds success: Two quasi-experimental interventions in the context of the Swiss aptitude test for medical-school admissions.

BACKGROUND: In Switzerland, the scholastic aptitude test for medical-school selection takes place in three languages. This study examined the effects of two quasi-experimental interventions that aimed to reduce existing differences in test results between the French- and German-speaking language candidates. METHODS: Between 2018 and 2023, the population of applicants to Swiss medical schools consisted of N = 18'824 German- and French-speaking individuals. Based on a quasi-experimental design, we examined the effects of two interventions regarding preparatory material, in these cohorts. The first intervention (2022) consisting of practice trials in baccalaureate schools in the canton of Fribourg enabled French-speaking candidates primarily from the canton of Fribourg to prepare more intensively with official tasks. Practice trials enable future candidates to complete a published test version under original conditions and thus prepare how to approach the real test. The second intervention (2023) released new preparatory material in all languages for one group of tasks for which differences between the language groups were more pronounced than in the other tasks. The test provider offered this material for free download together with existing preparation materials and thereby enabled more intensive preparation. RESULTS: After the first intervention, the initially small to medium-sized mean differences in z-transformed test scores between French-speaking candidates from Fribourg and German-speaking candidates were nearly eliminated (from 0.39 to 0.05). Also for French-speaking candidates from outside of the canton of Fribourg, the mean differences were smaller than before the intervention (0.48 before, 0.39 after first intervention). After the second intervention, particularly the mean differences in test scores between German-speaking and French-speaking candidates from outside of Fribourg were further reduced (to 0.24). CONCLUSIONS: The two interventions regarding material for preparing to participate in the aptitude test affected candidates' test scores. They reduced the gap between German- and French-speaking candidates showing that the additional benefits of commercial offers for test preparation are limited. Hence, offering comparable official preparation material to all language groups enhances test fairness.

Selective Classes and Early Health Inequalities in Comprehensive Schools in Finland.

BACKGROUND: The origin of inequalities in health outcomes has been explained by health selection and social causation models. Health selection processes operate particularly at school age. We study, if student allocation to teaching groups with aptitude tests (selective vs general class) differentiates adolescents by health behaviors and mental health. METHODS: Finnish schoolchildren 12-13 years from 12 selective classes, n = 248; 41 general classes, n = 703 answered a questionnaire on addictive products (tobacco, snus, alcohol, and energy drinks), digital media use, and mental health (health complaints, anxiety, and depression). Structural equation modeling was conducted to identify structures between outcomes, SEP (socioeconomic position), class type, and academic performance. RESULTS: Students in the selective classes reported less addictive digital media and addictive products use than students in the general classes. Differences in academic performance or SEP between the class types did not solely explain these differences. Mental health was not related to the class type. SEP was indirectly associated with health behaviors via the class type and academic performance. CONCLUSIONS: Selecting students to permanent teaching groups with aptitude tests differentiates students according to risky health behaviors. The impact of education policies using student grouping should also be evaluated in terms of students' health.

Validity and reliability of a repeated multi-changes of direction agility test in senior soccer players.

Background: This study aimed to evaluate the validity, reliability, and sensitivity of repeated multi-changes of direction agility test (rMCOD) compared to a soccer-specific field test of repeated sprint ability (S-RSA) and repeated sprint ability test (RSA). Methods: Thirty-five healthy male soccer players (age: 18.4 ± 1.3 years) from Tunisan national soccer league (elite and sub-elite) took part in this study. They performed the tests in a randomized order over five sessions interspaced by at least 72 h. The construct, predictive and discriminant validity, relative and absolute reliability, and sensitivity of the tests were analyzed. The total and best time of the test (the sum for all trials and the trial with the lowest duration on nine, six, and seven attempts for rMCOD, RSA, and S-RSA, respectively), fatigue index, rating of perceived exertion (RPE), and lactate concentration were recorded. Results: rMCOD correlated with both, S-RSA and RSA in total time (r = 0.85 and r = 0.52, respectively) and fatigue index (r = 0.74 and r = 0.83, respectively). Receiver operator characteristics were not able to discriminate between group levels (elite and sub-elite). When comparing training levels, only the fatigue index in S-RSA showed a difference between groups. Fatigue index, total time, and the best time in rMCOD showed excellent reliability, as well as the minimal change detectable (MCD = 0.89, MCD = 0.63, and MCD = 0.11, respectively) was higher than the standard error of the mean (SEM = 0.32, SEM = 0.23, and 0.04, respectively). Conclusion: rMCOD showed large to very large predictive validity compared with the S-RSA and RSA, being a reliable test for the following parameters: the best time and total time to perform the test. Nevertheless, this study design cannot ensure whether or not this test is able to detect real changes in performance in response to training since it did not include a training intervention; besides, rMCOD could not distinguish between elite and sub-elite players, which is a limitation.

Assessing the predictive validity of pre-admission criteria on dental students' academic performance: a cross-sectional study.

BACKGROUND: This study examines the multifaceted factors influencing academic performance among dental students in Saudi Arabia, exploring the complex relationships between demographic, educational, and psychological variables and students' Grade Point Averages (GPAs) to enhance global dental education admission strategies. METHODS: Data on demographics, academic performance indicators (including GPA, high school grades, test scores), accommodation type, parental education, suspension history, and English proficiency were collected through an English-language electronic questionnaire from 900 dental students across multiple institutions in Riyadh. The study employed Pearson's correlation analysis to examine relationships between GPA and various academic and demographic factors. A detailed regression analysis, using a stepwise selection based on Akaike's information criteria (AIC), identified significant GPA predictors and evaluated the average marginal effects of pre-admission variables on GPA, alongside rigorous diagnostic checks to validate the model's robustness. RESULTS: The study revealed a complex interplay of factors impacting GPA among dental students. High school grades, achievement, and aptitude test scores showed moderate positive correlations with GPA, while negative correlations were observed with age and number of times suspended. Regression analysis highlighted age, marital status, high school grades, and test scores as key predictors, with complex interaction effects demonstrating the layered influences of these factors. For instance, the negative impact of age on GPA was moderated by high school grade and aptitude test score. The analysis also highlighted the significant marginal effects of various pre-admission variables on GPA, such as the negative average impact of age and the positive impact of high school grades. CONCLUSIONS: This study provides valuable insights into the multifaceted determinants of academic success in dental education. Our findings underscore the significant roles of high school grades, achievement, and aptitude test scores, as well as the nuanced influence of age and marital status on GPA. These results advocate for a holistic approach in evaluating candidates for dental programs and have broader implications for global dental education, emphasizing the need for comprehensive admission strategies.

Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.

The purpose of this research is to demonstrate how using natural language processing (NLP) on narrative application data can improve prediction and reduce racial subgroup differences in scores used for selection decisions compared to mental ability test scores and numeric application data. We posit there is uncaptured and job-related constructs that can be gleaned from applicant text data using NLP. We test our hypotheses in an operational context across four samples (total N = 1,828) to predict selection into Officer Training School in the U.S. Air Force. Boards of three senior officers make selection decisions using a highly structured rating process based on mental ability tests, numeric application information (e.g., number of past jobs, college grades), and narrative application information (e.g., past job duties, achievements, interests, statements of objectives). Results showed that NLP scores of the narrative application generally (a) predict Board scores when combined with test scores and numeric application information at a level of correlation equivalent to the correlation between human raters (.60), (b) add incremental prediction of Board scores beyond mental ability tests and numeric application information, and (c) reduce subgroup differences between racial minorities and nonracial minorities in Board scores compared to mental ability tests and numeric application information. Moreover, NLP scores predict (a) job (training) performance, (b) job (training) performance beyond mental ability tests and numeric application information, and (c) even job (training) performance beyond Board scores. Scoring of narrative application data using NLP shows promise in addressing the validity-adverse impact dilemma in selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Effectiveness of a manual test at the beginning of dental school.

INTRODUCTION: This study analysed whether the SAZAMED (Self-assessment for dentistry studies) manual test conducted as an exercise or examination in the first year of dental studies, and the official aptitude test for medical studies (EMS), can predict future performance of dental students. MATERIALS AND METHODS: Students enrolled in the dental curriculum at the University of Basel between 2009/10 and 2019/20 were categorized: Group A conducted the SAZAMED in the third bachelor year (BA) and second master year (MA), group B practiced in the first and third BA and second MA and group C had to pass the test in the first BA with repetition in the third BA and second MA. SAZAMED comprised (i) wire bending, (ii) modelling sphere and cube, (iii) drawing mirror-inverted and (iv) indirect drawing over a mirror. Comparative variables were the EMS per cent rank and test scores in practical-clinical examinations in the third BA and the second MA. RESULTS: Data were available from 329 students over the 10-year period. Repeaters and dropouts had poorer performance in the first BA SAZAMED than students who did not (p < 0.001). Among the four SAZAMED exercises, mirror-inverted drawing was the strongest predictor of future student performance. The official aptitude test EMS correlated with the third BA certificate (p = 0.012), but failed to predict repeaters and dropouts. CONCLUSION: Since academic success was associated with significantly better scores on the first BA SAZAMED, it is considered a predictor and recommended in addition to the official EMS to identify students in need of intervention programmes or who should choose a different field of study.

Investigating correlations between the PAT and three hand-skill exercises to performance in preclinical laboratory courses.

OBJECTIVES: Perceptual ability test (PAT) is a valid determinant for spatial perceptions. However, proven psychomotor skill tests for dentistry do not currently exist. This study aimed to determine whether performance on the PAT, Operation game, mirror tracing, and wax carving exercises correlate to preclinical laboratory performance in Dental Anatomy and Restorative Dentistry courses. METHODS: First-year dental students (n = 96) participated in the study. Final laboratory grades were obtained from course directors of preclinical Restorative Dentistry and Dental Anatomy courses. Admissions committee provided participants' PAT results. Participants completed a wax carving exercise by carving a cube and a semicircle into a wax block using the wax subtraction method. Two calibrated faculty graded the carvings as Ideal (5), Satisfactory (3), or Unsatisfactory (1). Time to complete the Operation game and number of infractions were recorded. Participants traced the six-pointed star pattern on the Auto Scoring Mirror Tracer instrument in both clockwise and counterclockwise directions. Completion times and number of instances outside the pattern were recorded. Spearman Rank Correlations were used to determine associations at 0.05 confidence interval. RESULTS: Mean PAT score was 21.7; mean times to complete the Operation game and mirror tracing exercise were 4:20 and 1:30, respectively. Mean score on the wax carving exercise was 3.19. Correlations between the independent and dependent variables were minimal to weak. The wax carving exercise was the most reliable predictor of performance. CONCLUSIONS: When PAT scores are striated into low (less than 20), middle (21-22), and high (23-30), it was possible to predict performance in both preclinical laboratory courses.

Do students' personality traits change during medical training? A longitudinal cohort study.

Many medical schools incorporate assessments of personal characteristics, including personality traits, in their selection process. However, little is known about whether changes in personality traits during medical training affect the predictive validity of personality assessments. The present study addressed this issue by examining the stability of personality traits and their predictive validity over a 6-year medical training course. Participants were two cohorts of Swiss medical students (N = 272, 72% of students admitted to Year 2) from whom we collected demographic data, Swiss medical studies aptitude test (EMS) scores, Big Five personality traits scores measured at three times and scores on the multiple-choice and objective structured clinical examination parts of the final medical examination. Our findings indicated that personality traits had medium-to-high rank-order stability (r > .60 over 3 years and r > .50 over 6 years). Mean-level changes were moderate for agreeableness (d = + 0.72) and small for neuroticism and conscientiousness (d = -0.29, d = -0.25, respectively). Individual reliable change indices ranged from 4.5% for openness to 23.8% for neuroticism. The predictive validity was similar to that of the first three years of follow-up. To the best of our knowledge, this is the first study to investigate changes in personality across undergraduate curriculum. Medical students' personality traits were mostly stable across medical school and retain their predictive validity. Consequently, this study supports the use of tools measuring constructs underlying personality traits in selection. In addition, this study confirms that examination formats could favor students with certain personality traits.

Challenging conclusions about predictive bias against Hispanic test takers in personnel selection.

Berry et al. (2020) noted that predictive bias is a function of three factors: subgroup mean difference on the predictor (dx), subgroup mean difference on the criterion (dy), and test validity (rxy). They used meta-analytic estimates of each of these three to examine predictive bias against Hispanic test takers when cognitive tests are used in personnel selection. They found that tests underpredict Hispanic job performance by an average of .21 SDs, which would call into question the fairness of cognitive test use in personnel selection. We located 119 studies in which all three parameters-dy, dx, and rxy-could be obtained, thus holding sample, setting, and operationalization constant in estimating the three parameters within each study. This produced a substantially different conclusion: We find that tests overpredict Hispanic performance by .04-.20 SDs, depending on assumptions made about artifact corrections. Factors contributing to differences between the two studies include differences in range restriction corrections, sample incomparability, and Berry et al.'s use of rxy estimated from the total sample rather than within the majority subgroup. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Evidências de validade do teste informatizado para avaliação das funções executivas / Evidence of validity of the computerized test for evaluating executive functions / Evidencia de validez de la prueba computarizada para evaluación de funciones ejecutivas

Funções executivas (FE) referem-se ao conjunto de habilidades que, de forma integrada, coordenam o comportamento e a cognição. Assim, o comprometimento no desenvolvimento das FE está ligado a vários desfechos negativos ao longo da vida. Portanto, a avaliação dessas habilidades na infância é essencial para identificar e prevenir prejuízos na vida adulta. Este estudo teve como objetivo investigar evidências de validade do Teste Informatizado para Avaliação das Funções Executivas (Tafe) pelo critério de idade e pelo padrão de correlação entre medidas do TAFE e outras medidas de FE. Para isso, foi utilizada uma amostra de 51 crianças, entre 4 e 10 anos de idade, matriculadas em uma escola privada na cidade de Goiânia (GO), da pré-escola ao 4º ano do ensino fundamental. Como instrumentos, foram utilizados, além do Tafe, as tarefas de Bloco de Corsi, Teste de Trilhas A e B, Teste de Trilhas Pré-Escolares, Teste de Stroop Pré-Escolares e Subteste Dígitos da Escala Wechsler de Inteligência. Foram conduzidas análises estatísticas Kruskal Wallis para verificar a evidência de validade por relação com idade e análises de correlação não paramétrica de Spearman para avaliar as evidências de validade convergente. O instrumento investigado mostrou-se efetivo para discriminar entre as diferentes faixas etárias, assim, sensível ao desenvolvimento das FE. Os resultados obtidos no Tafe correlacionaram-se aos obtidos em outros testes que também avaliaram FE, mostrando claros padrões de convergência. Logo, as análises dos resultados fornecem evidências de validade ao Tafe, derivadas a partir de diferentes estratégias de investigação.(AU)

Executive functions (EF) refer to the set of skills that, in an integrated way, coordinate behavior and cognition. Thus, the commitment in developing EF is linked to several negative outcomes throughout life. Therefore, the assessment of these abilities in childhood is essential to identify and prevent possible harm in adult life. This study aimed to investigate evidences of validity of the Computerized Test for Evaluation of Executive Functions (TAFE) by the age criterion and by the pattern of correlation between TAFE measures and other EF measures. To this end, a sample of 51 children, between 4 and 10 years old, enrolled in a private school in the municipality of Goiânia (GO), from preschool to the 4th year of elementary school, was used. As instruments, in addition to TAFE, the tasks of Corsi Block, Trails Test A and B, Preschool Trails Test, Preschool Stroop Test and the Digits Subtest of the Wechsler Intelligence Scale were used. Kruskal Wallis statistical analyzes were conducted to verify the evidence of validity by age criteria and Spearman's nonparametric correlation analysis to assess evidence of convergent validation. The instrument investigated was effective to discriminate between different age groups, thus being sensitive to the development of EF. The results obtained in TAFE were correlated with those obtained in other tests that also assess FE, showing patterns of convergence. Therefore, the analysis of the results provides validation evidence to TAFE, derived from different investigation methods.(AU)

Las funciones ejecutivas (FE) se refieren al conjunto de habilidades que coordinan la conducta y la cognición. Así, el deterioro en el desarrollo de FE está relacionado con varios resultados negativos durante la vida. Por tanto, la valoración de estas habilidades en la infancia es fundamental para identificar y prevenir posibles daños en la vida adulta. Este estudio tuvo como objetivo investigar la evidencia de validez de la Prueba Computarizada para Evaluación de Funciones Ejecutivas (Tafe) por el criterio de edad y convergencia entre las medidas Tafe y otras medidas de FE. Para ello, se utilizó una muestra de 51 niños, de entre 4 y 10 años de edad, matriculados en un colegio privado de Goiânia (Goiás, Brasil) desde la preescolar hasta 4.º de primaria. Como instrumentos, además del Tafe, se utilizaron las tareas de Bloque de Corsi, el Test de los senderos A y B, el Test de los Senderos Preescolares, el Test de Stroop Preescolar y la Subprueba de Dígitos de la Escala Wechsler de Inteligencia. Se realizaron análisis estadísticos de Kruskal Wallis para verificar la validez por criterios de edad y análisis de correlación no-paramétrica de Spearman para evaluar la evidencia de validez convergente. Tafe demostró ser efectivo para discriminar entre diferentes grupos de edad, así es sensible al desarrollo de FE. Los resultados en Tafe se correlacionaron con los obtenidos en otras pruebas de FE, lo que muestra claros patrones de convergencia. Por tanto, el análisis aporta evidencias de validez para Tafe, derivadas de diferentes estrategias de investigación.(AU)

Reflexões sobre o trabalho on-line da neuropsicologia infantil numa visão histórico-cultural / Reflections on the online work of child neuropsychology from a cultural-historical perspective / Reflexiones sobre el trabajo online de la neuropsicología infantil desde una perspectiva histórico-cultural

O isolamento social provocado pela pandemia de covid-19 exigiu uma intensificação no estabelecimento de parâmetros para o trabalho do neuropsicólogo com suas especificidades. Assim, o objetivo deste artigo é levantar reflexões sobre a prática neuropsicológica infantil via internet baseada na nossa experiência prática vivenciada no período, relacionando com as referências teóricas da abordagem histórico-cutural e outros trabalhos que discorrem sobre as particularidades dessa modalidade de atendimento. Para isso, foi realizada uma pesquisa bibliográfica em fontes reconhecidas, como SciELO e Google Acadêmico, utilizando-se marcadores como "neuropsicologia on-line" e "teleneuropsicologia", entre outros, em português, inglês e espanhol. Os trabalhos encontrados foram utilizados como fundamento para discutir as ideias aqui apresentadas, a partir da experiência dos autores durante esse período. Considera-se que os neuropsicólogos precisam enfrentar a possível necessidade do trabalho on-line, considerando que as situações vividas durante a pandemia tendem a aumentar a necessidade de avaliações e intervenções neuropsicológicas. Ao mesmo tempo, se prevê que essa via de trabalho deve se intensificar após o isolamento, dentro do que se tem denominado como "o novo normal". Para isso, é essencial que haja esforço por parte da comunidade neuropsicológica para mostrar como essa área de trabalho pode ser útil para crianças, seus pais e professores. Conclui-se que é preciso ter a noção de que nenhuma via ou metodologia de trabalho é absolutamente completa, sendo essencial a instrumentalização das vantagens e desvantagens de todas elas.(AU)

The social isolation caused by the COVID-19 pandemic demanded intensification in the establishment of parameters for the neuropsychologist's work with its specificities. Thus, the aim of this article is to raise reflections on child neuropsychological practice via the internet, based on our practical experience in this period, relating to the theoretical references of the cultural-historical approach and other works that discuss the particularities of this type of care. For this, a bibliographic research was carried out in recognized sources such as SciELO and Google Scholar, using markers such as "online neuropsychology" and teleneuropsychology, among others, in Portuguese, English and Spanish. The works found were used as a basis for discussing the ideas presented here which are based on the authors' experience during this period. It is considered that neuropsychologists need to face the possible need for online work, considering that situations experienced during the pandemic tend to increase the need for neuropsychological assessments and interventions. At the same time, it is expected that this work path should intensify after isolation, within of what has been called "the new normal". For this, an effort by the neuropsychological community is essential to show how this area of work can be useful for children, their parents and teachers. It is concluded that it is necessary to have the notion that no way or work methodology is absolutely complete, being essential the instrumentalization of the advantages and disadvantages of all of them.(AU)

El aislamiento social provocado por la pandemia de Covid-19 exigió una intensificación en el establecimiento de parámetros para el trabajo del neuropsicólogo con sus especificidades. Así, el objetivo de este artículo es plantear reflexiones sobre la práctica neuropsicológica infantil a través de internet, a partir de nuestra experiencia práctica en este período, en relación con los referentes teóricos del enfoque histórico-cultural y otros trabajos que discuten las particularidades de este tipo de cuidado. Para esto, se realizó una investigación bibliográfica en fuentes reconocidas como SciELO y Google Académico, utilizando marcadores como "neuropsicología en línea" y teleneuropsicología, entre otros, en portugués, inglés y español. Los trabajos encontrados sirvieron de base para discutir las ideas aquí presentadas a partir de la experiencia de los autores durante este período. Se considera que los neuropsicólogos necesitan enfrentar la posible necesidad de trabajo en línea, considerando que las situaciones vividas durante la pandemia tienden a incrementar la necesidad de evaluaciones e intervenciones neuropsicológicas. Al mismo tiempo, se espera que esta vía de trabajo se intensifique después del aislamiento, dentro de lo que se ha denominado "la nueva normalidad". Para esto, un esfuerzo de la comunidad neuropsicológica es fundamental para mostrar cómo esta área de trabajo puede ser útil para los niños, sus padres y profesores. Se concluye que es necesario tener la noción de que ninguna vía o metodología de trabajo es absolutamente completa, siendo imprescindible la instrumentalización de las ventajas y desventajas de todas ellas.(AU)

20 anos da comissão consultiva em avaliação psicológica: desafios da ADI 3481 / 20 years of the advisory committee on psychological assessment: challenges of ADI 3481 / 20 años de la comisión consultiva en evaluación psicológica: desafíos de la ADI 3481

A Comissão Consultiva em Avaliação Psicológica (CCAP) do Conselho Federal de Psicologia (CFP), em seu 20º aniversário, vem discutir os possíveis efeitos, ainda efetivamente desconhecidos, da Ação Direta de Inconstitucionalidade (ADI) 3481, instruída no Supremo Tribunal Federal (STF), a qual desestruturou o modo como os testes psicológicos eram comercializados no Brasil. A livre comercialização de testes psicológicos coloca em risco a segurança de avaliações psicológicas e cabe à categoria profissional pensar estratégias de enfrentamento desses riscos. Neste artigo, são discutidos possíveis efeitos da ADI 3481 para a categoria profissional da psicologia, bem como para a sociedade em geral, e são também elencadas possíveis estratégias de enfrentamento desses riscos, sem desconsiderar aspectos éticos relacionados a eles. Dessa forma, busca-se neste manuscrito, além da problematização dos efeitos derivados da ADI 3481, pensar soluções ou alternativas que venham a redirecionar a trajetória da área da avaliação psicológica no Brasil. Com isso, abre-se um espaço de discussão e encaminhamentos que a categoria profissional precisará tomar nos próximos anos.(AU)

The Advisory Commission for Psychological Assessment of the Federal Council of Psychology discusses, on its 20th anniversary, the possible and still effectively unknown effects of the Direct Action of Unconstitutionality (DAU) 3481, following the Supreme Federal Court, which interrupted how psychological tests were marketed in Brazil. The free trade of psychological tests puts the safety of psychological assessments at risk, and this professional category must think of strategies to face these risks. This study discusses the possible effects of DAU 3481 for professional psychology and for society in general, listing possible strategies for coping with these risks without disregarding its ethical aspects. Thus, this study seeks to problematize the effects derived from DAU 3481 and think of solutions or alternatives that may redirect the trajectory of the field of psychological assessment in Brazil, thus opening a space for discussion and referrals professional psychology will require in the coming years.(AU)

La Comisión Consultiva en Evaluación Psicológica (CCEP) del Consejo Federal de Psicología (CFP), en su 20.º aniversario, propone discutir los posibles efectos aún efectivamente desconocidos de la Acción Directa de Inconstitucionalidad (ADI) 3481, determinada por el Supremo Tribunal Federal (STF), por la cual trastornó la forma de comercializar las pruebas psicológicas en Brasil. La comercialización sin restricciones de las pruebas psicológicas pone en riesgo la seguridad de las evaluaciones psicológicas, y le corresponde a la categoría profesional pensar estrategias para enfrentar estos riesgos. En este artículo se discuten los posibles efectos de la ADI 3481 para la categoría profesional de la Psicología, así como para la sociedad en general, pero también se enumeran posibles estrategias para el enfrentamiento de estos riesgos, sin descuidar los aspectos éticos relacionados con ellos. Así, este manuscrito busca, además de problematizar los efectos derivados de la ADI 3481, pensar en soluciones o alternativas que puedan reconducir la trayectoria del campo de la evaluación psicológica en Brasil. Esto abre un espacio de discusión y derivaciones que la categoría profesional deberá tomar en los próximos años.(AU)

Sistema de avaliação de testes psicológicos: histórias contadas e não contadas / Psychological test evaluation system: stories told and untold / Sistema de evaluación de tests psicológicos: historias contadas y no contadas

O Sistema de Avaliação de Testes Psicológicos (SATEPSI) recebeu notoriedade entre brasileiros e estrangeiros por oferecer um complexo sistema de qualificação dos testes psicológicos, pouco visto em âmbito mundial. Sua elaboração dependeu de uma autarquia, que o financiou, normatizou e o mantém, mas também de pesquisadores docentes de avaliação psicológica, que trouxeram a expertise da área para que houvesse o pleno estabelecimento de seus parâmetros. Passadas duas décadas de seu lançamento, o SATEPSI foi tema de artigos, capítulos, lives e diálogos digitais, nos quais foram destaque, de modo geral, as Resoluções do Conselho Federal de Psicologia, que o normatiza, e seus impactos para a área de avaliação psicológica - como, por exemplo, o aumento do número de pesquisas e de testes brasileiros qualificados. O que se pretende neste artigo é mencionar sua construção, à luz dos autores que vivenciaram o SATEPSI em funções e tempos distintos. Atenção especial será dada aos Métodos Projetivos, cuja história ainda é pouco revelada.(AU)

The system to evaluate psychological tests (Satepsi) received notoriety among Brazilians and foreigners for offering a complex system of qualification of psychological tests, which is rarely seen worldwide. Its development depended on an autarchy (which financed, standardized, and maintains it) and on researchers teaching psychological assessment, who brought their expertise to the area so its parameters could be fully established. After two decades of its launch, Satepsi was the subject of articles, chapters, lives, and digital dialogues, which usually highlighted the Resolutions of the Federal Council of Psychology that normatize psychological evaluation and their impacts, such as the increase in the number of qualified Brazilian tests. This study aims to mention its construction in the light of the authors who experienced Satepsi in different functions and times, giving special attention to Projective Methods, whose history remains to be shown.(AU)

El Sistema de Evaluación de Tests Psicológicos (SATEPSI) ganó notoriedad entre los brasileños y los extranjeros por ofrecer un complejo sistema de calificación de los tests psicológicos, poco frecuente a nivel mundial. Su elaboración dependió de una autarquía, que lo financió, lo estandarizó y lo mantiene, pero también de investigadores docentes de evaluación psicológica, que trajeron la experiencia del área para que hubiera el pleno establecimiento de sus parámetros. Tras dos décadas de su lanzamiento, SATEPSI fue tema de artículos, capítulos, en directo y diálogos digitales, en los cuales destacaron, de modo general, las Resoluciones del Consejo Federal de Psicología que lo normatiza y sus impactos para el área de evaluación psicológica, como el aumento del número de investigaciones y de pruebas brasileñas calificadas. Lo que se pretende en este artículo es mencionar su construcción, a la luz de los autores que vivieron el SATEPSI en funciones y tiempos distintos. Se prestará especial atención a los métodos proyectivos cuya historia aún no se ha revelado.(AU)

An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study.

BACKGROUND: Previous studies have shown performance in the University Clinical Aptitude Test (UCAT) to be associated with measures of candidate socio-economic advantage such as parental occupation and type of school attended. It is possible that access to preparation support and materials may in part explain these associations. In this paper we determine whether use of formal preparation resources is associated with higher UCAT scores and whether differences in use of preparation resources exist between socio-demographic groups. METHODS: After completing the 2017 UCAT UK school-leaver candidates (n = 14,332) were asked to answer a questionnaire regarding their use of official UCAT and commercial resources, school-based support, and time spent preparing. Multiple linear and logistic regression models were used to evaluate the associations between preparedness, demographic characteristics and UCAT performance. RESULTS: Five thousand, four hundred thirty-nine (38%) candidates responded to the questionnaire. Use of freely available UCAT official practice tests, paid commercial materials, attendance at school-based preparation courses and spending more time preparing were significantly associated with higher UCAT scores. Candidates who were from less deprived backgrounds and attending independent or grammar schools were significantly more likely to use paid commercial materials and spend longer preparing. CONCLUSIONS: Reported use of preparation resources varies between candidates from different socio-demographic backgrounds and is associated independently with performance in the UCAT. Increasing the availability of freely available resources may mitigate some of these differences.

Does Inclusion of Mental Agility Test in Admission Exam Help Better Student Selection in the Health Professions Education Programs?

Academic ability test has been used predominantly in student selection of medical and allied health profession education programs in Nepal. But the use of academic performance as the single selection criterion puts the students from low socioeconomic background at greater disadvantage despite equal suitability due to the lack of adequate guidance and support during their schooling. To address this limitation, use of aptitude test i.e. both the general cognitive and non-cognitive ability tests that measures fluid intelligence and personality traits respectively has been practiced. The measurement of non-cognitive traits has been found to predict the clinical examination score. In Nepal, for the first time, Patan Academy of Health Sciences implemented the assessment of aptitude test (both cognitive and non-cognitive ability test) for the student selection in the undergraduate medical program. Since the inception, Medical Education Commission in Nepal embraced Mental Agility Test, a component of an aptitude test, along with the academic ability test for the nationwide common entrance test in all the undergraduate Health Professions Education Programs. This indeed is an innovative approach in student selection, but in the context of Nepal whether the use of these tools is appropriate in the entrance exam requires psychometric evaluation and further validation through graduates' performance after their enrolment. Keywords: Health professions education; mental agility test; student selection.

BMAT's predictive validity for medical school performance: A retrospective cohort study.

BACKGROUND: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. METHODS: This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students. RESULTS: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades. DISCUSSION: Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Association of admissions factors with student performance in preclinical operative dentistry.

PURPOSE/OBJECTIVE: Admission to dental school is based on various factors including academic achievements in undergraduate coursework and the Dental Admission Test (DAT) scores. Students' success in an operative course requires fundamental knowledge, hand skills, spatial awareness, and self-assessment ability. The goal of this study is to evaluate how admissions factors, such as Grade Point Average (GPA) and DAT, including the Perceptual Ability Test (PAT), relate to students' academic and preclinical performance and self-assessment skills in preclinical operative dentistry. METHODS: A total of 239 students were included from seven class years (2016-2022). Third-year dental students participated in a preclinical operative dentistry course. At the end of the course, they took the final multiple-choice exam and performed four competency examination procedures: Class II amalgam preparation and restoration and Class III resin-composite preparation and restoration. Calibrated faculty graded students' work independently and students also self-assessed their performance using the same rubrics as faculty. Linear regressions were performed to estimate the association between the admission factors with the mean faculty scores (measuring preclinical performance), student-faculty (S-F) gap scores (evaluating self-assessment skills), and their final didactic exam scores. RESULTS: Overall, students' self-assessment was higher compared to faculty score. Linear regression analysis demonstrated positive correlations between the PAT and students' preclinical performance as well as between the DAT and their didactic exam scores. In general, S-F gap score decreased as PAT score increased, and it was statistically significant lower for the Class III preparation, indicating a better self-assessment skill. No correlations were observed between student performance and GPA scores. CONCLUSION: The findings from the association between student performance and admission factors may play an important role in the dental school admissions process and assist students who may benefit from early faculty intervention and support.

Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002-2018 (UKMED54).

OBJECTIVES: To examine factors associated with declaration of disability by medical students and doctors, and the association of declared disability with academic performance. DESIGN: Observational study using record-linked data collected between 2002 and 2018. SETTING: UK Medical Education Database is a repository of data relating to training of medical students and doctors. Disability and other data are record-linked. PARTICIPANTS: All students starting at a UK medical school between 2002 and 2018 (n=135 930). MAIN OUTCOME MEASURES: Declared disability was categorised by the Higher Education Statistics Authority. Outcomes related to undergraduate academic performance included scores in the educational performance measure (EPM), prescribing safety assessment and situational judgement test. Performance in postgraduate examinations was studied, as well as prior attainment in school examinations and aptitude tests. RESULTS: Specific learning disability (SLD) was the most commonly declared disability (3.5% compared with the next most commonly declared disability at 1.0% of n=129 345 all cases in the study), and during the period covered by the data, SLD declarations increased from 1.4% (n=6440 for students starting in 2002) to 4.6% (n=8625 for students starting in 2018). In a logistic regression, the following factors predicted recording of SLD on entry to medical school ((exp(B)±95% CI), p<0.0001 unless otherwise stated): attendance at a fee-paying school (2.306±0.178), graduate status (1.806±0.205), participation of local areas quintile (1.089±0.030), age (1.034±0.012). First year medical students were less likely to declare SLD if they were from a non-white ethnic background (Asian/Asian British 0.324±0.034, black/black British 0.571±0.102, mixed 0.731±0.108, other ethnic groups 0.566±0.120), female (0.913±0.059; p=0.007) or from a low index of multiple deprivation quintile (0.963±0.029); p=0.017. In univariate analysis with Bonferroni corrections applied for multiple tests, no significant difference was observed in the recording of SLD according to socioeconomic class (χ2=5.637, p=1), whether or not a student's parents had a higher education (χ2=0.140, p=1), or whether or not a student had received a United Kingdom Clinical Aptitude Test (UKCAT) bursary (χ2=7.661, p=0.068). Students who declared SLD at some point in medical school (n=4830) had lower EPM normalised deviate values (-0.390) than those who did not (-0.119) (F=189.872, p<0.001). Those for whom SLD was recorded were as likely to complete the course successfully as those who did not declare disability (93.0% successful completion by those for whom SLD declared from year 1 (n=2480), 92.2% by those for whom SLD declared after year 1 (n=2350), 91.6% by those for whom SD not declared at any point (n=85 180)) (χ2=6.905, p=0.032). Of 3580 first year students who declared SLD, 43.1% had not sat the UKCAT Special Educational Needs aptitude test (which gives extra time for those with special educational needs), while 28% of 2400 registrants for whom SLD was recorded as medical students did not declare it at General Medical Council registration. CONCLUSIONS: Substantial increases in declaration of SLD may reflect changes in the social and legal environment during the period of the study. Those who declare SLD are just as likely to gain a primary medical qualification as those who do not. For some individuals, disability declaration appears to depend on context, based on differences in numbers declaring SLD before, during and after medical school.

Situational judgement test performance and subsequent misconduct in medical students.

INTRODUCTION: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. METHODS: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. RESULTS: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect-size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. CONCLUSIONS: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection.