Test preparation breeds success: Two quasi-experimental interventions in the context of the Swiss aptitude test for medical-school admissions.

BACKGROUND: In Switzerland, the scholastic aptitude test for medical-school selection takes place in three languages. This study examined the effects of two quasi-experimental interventions that aimed to reduce existing differences in test results between the French- and German-speaking language candidates. METHODS: Between 2018 and 2023, the population of applicants to Swiss medical schools consisted of N = 18'824 German- and French-speaking individuals. Based on a quasi-experimental design, we examined the effects of two interventions regarding preparatory material, in these cohorts. The first intervention (2022) consisting of practice trials in baccalaureate schools in the canton of Fribourg enabled French-speaking candidates primarily from the canton of Fribourg to prepare more intensively with official tasks. Practice trials enable future candidates to complete a published test version under original conditions and thus prepare how to approach the real test. The second intervention (2023) released new preparatory material in all languages for one group of tasks for which differences between the language groups were more pronounced than in the other tasks. The test provider offered this material for free download together with existing preparation materials and thereby enabled more intensive preparation. RESULTS: After the first intervention, the initially small to medium-sized mean differences in z-transformed test scores between French-speaking candidates from Fribourg and German-speaking candidates were nearly eliminated (from 0.39 to 0.05). Also for French-speaking candidates from outside of the canton of Fribourg, the mean differences were smaller than before the intervention (0.48 before, 0.39 after first intervention). After the second intervention, particularly the mean differences in test scores between German-speaking and French-speaking candidates from outside of Fribourg were further reduced (to 0.24). CONCLUSIONS: The two interventions regarding material for preparing to participate in the aptitude test affected candidates' test scores. They reduced the gap between German- and French-speaking candidates showing that the additional benefits of commercial offers for test preparation are limited. Hence, offering comparable official preparation material to all language groups enhances test fairness.

Insights into undergraduate medical student selection tools: a systematic review and meta-analysis.

PURPOSE: Evaluating medical school selection tools is vital for evidence-based student selection. With previous reviews revealing knowledge gaps, this meta-analysis offers insights into the effectiveness of these selection tools. METHODS: A systematic review and meta-analysis were conducted applying the following criteria: peer-reviewed articles available in English, published from 2010 and which include empirical data linking performance in selection tools with assessment and dropout outcomes of undergraduate entry medical programs. Systematic reviews, meta-analyses, general opinion pieces, or commentaries were excluded. Effect sizes (ESs) of the predictability of academic and clinical performance within and by the end of the medicine program were extracted, and the pooled ESs were presented. RESULTS: Sixty-seven out of 2,212 articles were included, which yielded 236 ESs. Previous academic achievement predicted medical program academic performance (Cohen's d=0.697 in early program; 0.619 in end of program) and clinical exams (0.545 in end of program). Within aptitude tests, verbal reasoning and quantitative reasoning predicted academic achievement in the early program and in the last years (0.704 & 0.643, respectively). Overall aptitude tests predicted academic achievement in both the early and last years (0.550 & 0.371, respectively). Neither panel interviews, multiple mini-interviews, nor situational judgement tests (SJT) yielded statistically significant pooled ES. CONCLUSION: Current evidence suggests that learning outcomes are predicted by previous academic achievement and aptitude tests. The predictive value of SJT and topics such as selection algorithms, features of interview (e.g., content of the questions) and the way the interviewers' reports are used, warrant further research.

Selective Classes and Early Health Inequalities in Comprehensive Schools in Finland.

BACKGROUND: The origin of inequalities in health outcomes has been explained by health selection and social causation models. Health selection processes operate particularly at school age. We study, if student allocation to teaching groups with aptitude tests (selective vs general class) differentiates adolescents by health behaviors and mental health. METHODS: Finnish schoolchildren 12-13 years from 12 selective classes, n = 248; 41 general classes, n = 703 answered a questionnaire on addictive products (tobacco, snus, alcohol, and energy drinks), digital media use, and mental health (health complaints, anxiety, and depression). Structural equation modeling was conducted to identify structures between outcomes, SEP (socioeconomic position), class type, and academic performance. RESULTS: Students in the selective classes reported less addictive digital media and addictive products use than students in the general classes. Differences in academic performance or SEP between the class types did not solely explain these differences. Mental health was not related to the class type. SEP was indirectly associated with health behaviors via the class type and academic performance. CONCLUSIONS: Selecting students to permanent teaching groups with aptitude tests differentiates students according to risky health behaviors. The impact of education policies using student grouping should also be evaluated in terms of students' health.

Validity and reliability of a repeated multi-changes of direction agility test in senior soccer players.

Background: This study aimed to evaluate the validity, reliability, and sensitivity of repeated multi-changes of direction agility test (rMCOD) compared to a soccer-specific field test of repeated sprint ability (S-RSA) and repeated sprint ability test (RSA). Methods: Thirty-five healthy male soccer players (age: 18.4 ± 1.3 years) from Tunisan national soccer league (elite and sub-elite) took part in this study. They performed the tests in a randomized order over five sessions interspaced by at least 72 h. The construct, predictive and discriminant validity, relative and absolute reliability, and sensitivity of the tests were analyzed. The total and best time of the test (the sum for all trials and the trial with the lowest duration on nine, six, and seven attempts for rMCOD, RSA, and S-RSA, respectively), fatigue index, rating of perceived exertion (RPE), and lactate concentration were recorded. Results: rMCOD correlated with both, S-RSA and RSA in total time (r = 0.85 and r = 0.52, respectively) and fatigue index (r = 0.74 and r = 0.83, respectively). Receiver operator characteristics were not able to discriminate between group levels (elite and sub-elite). When comparing training levels, only the fatigue index in S-RSA showed a difference between groups. Fatigue index, total time, and the best time in rMCOD showed excellent reliability, as well as the minimal change detectable (MCD = 0.89, MCD = 0.63, and MCD = 0.11, respectively) was higher than the standard error of the mean (SEM = 0.32, SEM = 0.23, and 0.04, respectively). Conclusion: rMCOD showed large to very large predictive validity compared with the S-RSA and RSA, being a reliable test for the following parameters: the best time and total time to perform the test. Nevertheless, this study design cannot ensure whether or not this test is able to detect real changes in performance in response to training since it did not include a training intervention; besides, rMCOD could not distinguish between elite and sub-elite players, which is a limitation.

Assessing the predictive validity of pre-admission criteria on dental students' academic performance: a cross-sectional study.

BACKGROUND: This study examines the multifaceted factors influencing academic performance among dental students in Saudi Arabia, exploring the complex relationships between demographic, educational, and psychological variables and students' Grade Point Averages (GPAs) to enhance global dental education admission strategies. METHODS: Data on demographics, academic performance indicators (including GPA, high school grades, test scores), accommodation type, parental education, suspension history, and English proficiency were collected through an English-language electronic questionnaire from 900 dental students across multiple institutions in Riyadh. The study employed Pearson's correlation analysis to examine relationships between GPA and various academic and demographic factors. A detailed regression analysis, using a stepwise selection based on Akaike's information criteria (AIC), identified significant GPA predictors and evaluated the average marginal effects of pre-admission variables on GPA, alongside rigorous diagnostic checks to validate the model's robustness. RESULTS: The study revealed a complex interplay of factors impacting GPA among dental students. High school grades, achievement, and aptitude test scores showed moderate positive correlations with GPA, while negative correlations were observed with age and number of times suspended. Regression analysis highlighted age, marital status, high school grades, and test scores as key predictors, with complex interaction effects demonstrating the layered influences of these factors. For instance, the negative impact of age on GPA was moderated by high school grade and aptitude test score. The analysis also highlighted the significant marginal effects of various pre-admission variables on GPA, such as the negative average impact of age and the positive impact of high school grades. CONCLUSIONS: This study provides valuable insights into the multifaceted determinants of academic success in dental education. Our findings underscore the significant roles of high school grades, achievement, and aptitude test scores, as well as the nuanced influence of age and marital status on GPA. These results advocate for a holistic approach in evaluating candidates for dental programs and have broader implications for global dental education, emphasizing the need for comprehensive admission strategies.

Using natural language processing to increase prediction and reduce subgroup differences in personnel selection decisions.

The purpose of this research is to demonstrate how using natural language processing (NLP) on narrative application data can improve prediction and reduce racial subgroup differences in scores used for selection decisions compared to mental ability test scores and numeric application data. We posit there is uncaptured and job-related constructs that can be gleaned from applicant text data using NLP. We test our hypotheses in an operational context across four samples (total N = 1,828) to predict selection into Officer Training School in the U.S. Air Force. Boards of three senior officers make selection decisions using a highly structured rating process based on mental ability tests, numeric application information (e.g., number of past jobs, college grades), and narrative application information (e.g., past job duties, achievements, interests, statements of objectives). Results showed that NLP scores of the narrative application generally (a) predict Board scores when combined with test scores and numeric application information at a level of correlation equivalent to the correlation between human raters (.60), (b) add incremental prediction of Board scores beyond mental ability tests and numeric application information, and (c) reduce subgroup differences between racial minorities and nonracial minorities in Board scores compared to mental ability tests and numeric application information. Moreover, NLP scores predict (a) job (training) performance, (b) job (training) performance beyond mental ability tests and numeric application information, and (c) even job (training) performance beyond Board scores. Scoring of narrative application data using NLP shows promise in addressing the validity-adverse impact dilemma in selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Effectiveness of a manual test at the beginning of dental school.

INTRODUCTION: This study analysed whether the SAZAMED (Self-assessment for dentistry studies) manual test conducted as an exercise or examination in the first year of dental studies, and the official aptitude test for medical studies (EMS), can predict future performance of dental students. MATERIALS AND METHODS: Students enrolled in the dental curriculum at the University of Basel between 2009/10 and 2019/20 were categorized: Group A conducted the SAZAMED in the third bachelor year (BA) and second master year (MA), group B practiced in the first and third BA and second MA and group C had to pass the test in the first BA with repetition in the third BA and second MA. SAZAMED comprised (i) wire bending, (ii) modelling sphere and cube, (iii) drawing mirror-inverted and (iv) indirect drawing over a mirror. Comparative variables were the EMS per cent rank and test scores in practical-clinical examinations in the third BA and the second MA. RESULTS: Data were available from 329 students over the 10-year period. Repeaters and dropouts had poorer performance in the first BA SAZAMED than students who did not (p < 0.001). Among the four SAZAMED exercises, mirror-inverted drawing was the strongest predictor of future student performance. The official aptitude test EMS correlated with the third BA certificate (p = 0.012), but failed to predict repeaters and dropouts. CONCLUSION: Since academic success was associated with significantly better scores on the first BA SAZAMED, it is considered a predictor and recommended in addition to the official EMS to identify students in need of intervention programmes or who should choose a different field of study.

Investigating correlations between the PAT and three hand-skill exercises to performance in preclinical laboratory courses.

OBJECTIVES: Perceptual ability test (PAT) is a valid determinant for spatial perceptions. However, proven psychomotor skill tests for dentistry do not currently exist. This study aimed to determine whether performance on the PAT, Operation game, mirror tracing, and wax carving exercises correlate to preclinical laboratory performance in Dental Anatomy and Restorative Dentistry courses. METHODS: First-year dental students (n = 96) participated in the study. Final laboratory grades were obtained from course directors of preclinical Restorative Dentistry and Dental Anatomy courses. Admissions committee provided participants' PAT results. Participants completed a wax carving exercise by carving a cube and a semicircle into a wax block using the wax subtraction method. Two calibrated faculty graded the carvings as Ideal (5), Satisfactory (3), or Unsatisfactory (1). Time to complete the Operation game and number of infractions were recorded. Participants traced the six-pointed star pattern on the Auto Scoring Mirror Tracer instrument in both clockwise and counterclockwise directions. Completion times and number of instances outside the pattern were recorded. Spearman Rank Correlations were used to determine associations at 0.05 confidence interval. RESULTS: Mean PAT score was 21.7; mean times to complete the Operation game and mirror tracing exercise were 4:20 and 1:30, respectively. Mean score on the wax carving exercise was 3.19. Correlations between the independent and dependent variables were minimal to weak. The wax carving exercise was the most reliable predictor of performance. CONCLUSIONS: When PAT scores are striated into low (less than 20), middle (21-22), and high (23-30), it was possible to predict performance in both preclinical laboratory courses.

Do students' personality traits change during medical training? A longitudinal cohort study.

Many medical schools incorporate assessments of personal characteristics, including personality traits, in their selection process. However, little is known about whether changes in personality traits during medical training affect the predictive validity of personality assessments. The present study addressed this issue by examining the stability of personality traits and their predictive validity over a 6-year medical training course. Participants were two cohorts of Swiss medical students (N = 272, 72% of students admitted to Year 2) from whom we collected demographic data, Swiss medical studies aptitude test (EMS) scores, Big Five personality traits scores measured at three times and scores on the multiple-choice and objective structured clinical examination parts of the final medical examination. Our findings indicated that personality traits had medium-to-high rank-order stability (r > .60 over 3 years and r > .50 over 6 years). Mean-level changes were moderate for agreeableness (d = + 0.72) and small for neuroticism and conscientiousness (d = -0.29, d = -0.25, respectively). Individual reliable change indices ranged from 4.5% for openness to 23.8% for neuroticism. The predictive validity was similar to that of the first three years of follow-up. To the best of our knowledge, this is the first study to investigate changes in personality across undergraduate curriculum. Medical students' personality traits were mostly stable across medical school and retain their predictive validity. Consequently, this study supports the use of tools measuring constructs underlying personality traits in selection. In addition, this study confirms that examination formats could favor students with certain personality traits.

Challenging conclusions about predictive bias against Hispanic test takers in personnel selection.

Berry et al. (2020) noted that predictive bias is a function of three factors: subgroup mean difference on the predictor (dx), subgroup mean difference on the criterion (dy), and test validity (rxy). They used meta-analytic estimates of each of these three to examine predictive bias against Hispanic test takers when cognitive tests are used in personnel selection. They found that tests underpredict Hispanic job performance by an average of .21 SDs, which would call into question the fairness of cognitive test use in personnel selection. We located 119 studies in which all three parameters-dy, dx, and rxy-could be obtained, thus holding sample, setting, and operationalization constant in estimating the three parameters within each study. This produced a substantially different conclusion: We find that tests overpredict Hispanic performance by .04-.20 SDs, depending on assumptions made about artifact corrections. Factors contributing to differences between the two studies include differences in range restriction corrections, sample incomparability, and Berry et al.'s use of rxy estimated from the total sample rather than within the majority subgroup. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study.

BACKGROUND: Previous studies have shown performance in the University Clinical Aptitude Test (UCAT) to be associated with measures of candidate socio-economic advantage such as parental occupation and type of school attended. It is possible that access to preparation support and materials may in part explain these associations. In this paper we determine whether use of formal preparation resources is associated with higher UCAT scores and whether differences in use of preparation resources exist between socio-demographic groups. METHODS: After completing the 2017 UCAT UK school-leaver candidates (n = 14,332) were asked to answer a questionnaire regarding their use of official UCAT and commercial resources, school-based support, and time spent preparing. Multiple linear and logistic regression models were used to evaluate the associations between preparedness, demographic characteristics and UCAT performance. RESULTS: Five thousand, four hundred thirty-nine (38%) candidates responded to the questionnaire. Use of freely available UCAT official practice tests, paid commercial materials, attendance at school-based preparation courses and spending more time preparing were significantly associated with higher UCAT scores. Candidates who were from less deprived backgrounds and attending independent or grammar schools were significantly more likely to use paid commercial materials and spend longer preparing. CONCLUSIONS: Reported use of preparation resources varies between candidates from different socio-demographic backgrounds and is associated independently with performance in the UCAT. Increasing the availability of freely available resources may mitigate some of these differences.

Does Inclusion of Mental Agility Test in Admission Exam Help Better Student Selection in the Health Professions Education Programs?

Academic ability test has been used predominantly in student selection of medical and allied health profession education programs in Nepal. But the use of academic performance as the single selection criterion puts the students from low socioeconomic background at greater disadvantage despite equal suitability due to the lack of adequate guidance and support during their schooling. To address this limitation, use of aptitude test i.e. both the general cognitive and non-cognitive ability tests that measures fluid intelligence and personality traits respectively has been practiced. The measurement of non-cognitive traits has been found to predict the clinical examination score. In Nepal, for the first time, Patan Academy of Health Sciences implemented the assessment of aptitude test (both cognitive and non-cognitive ability test) for the student selection in the undergraduate medical program. Since the inception, Medical Education Commission in Nepal embraced Mental Agility Test, a component of an aptitude test, along with the academic ability test for the nationwide common entrance test in all the undergraduate Health Professions Education Programs. This indeed is an innovative approach in student selection, but in the context of Nepal whether the use of these tools is appropriate in the entrance exam requires psychometric evaluation and further validation through graduates' performance after their enrolment. Keywords: Health professions education; mental agility test; student selection.

BMAT's predictive validity for medical school performance: A retrospective cohort study.

BACKGROUND: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. METHODS: This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students. RESULTS: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades. DISCUSSION: Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Association of admissions factors with student performance in preclinical operative dentistry.

PURPOSE/OBJECTIVE: Admission to dental school is based on various factors including academic achievements in undergraduate coursework and the Dental Admission Test (DAT) scores. Students' success in an operative course requires fundamental knowledge, hand skills, spatial awareness, and self-assessment ability. The goal of this study is to evaluate how admissions factors, such as Grade Point Average (GPA) and DAT, including the Perceptual Ability Test (PAT), relate to students' academic and preclinical performance and self-assessment skills in preclinical operative dentistry. METHODS: A total of 239 students were included from seven class years (2016-2022). Third-year dental students participated in a preclinical operative dentistry course. At the end of the course, they took the final multiple-choice exam and performed four competency examination procedures: Class II amalgam preparation and restoration and Class III resin-composite preparation and restoration. Calibrated faculty graded students' work independently and students also self-assessed their performance using the same rubrics as faculty. Linear regressions were performed to estimate the association between the admission factors with the mean faculty scores (measuring preclinical performance), student-faculty (S-F) gap scores (evaluating self-assessment skills), and their final didactic exam scores. RESULTS: Overall, students' self-assessment was higher compared to faculty score. Linear regression analysis demonstrated positive correlations between the PAT and students' preclinical performance as well as between the DAT and their didactic exam scores. In general, S-F gap score decreased as PAT score increased, and it was statistically significant lower for the Class III preparation, indicating a better self-assessment skill. No correlations were observed between student performance and GPA scores. CONCLUSION: The findings from the association between student performance and admission factors may play an important role in the dental school admissions process and assist students who may benefit from early faculty intervention and support.

Factors associated with declaration of disability in medical students and junior doctors, and the association of declared disability with academic performance: observational study using data from the UK Medical Education Database, 2002-2018 (UKMED54).

OBJECTIVES: To examine factors associated with declaration of disability by medical students and doctors, and the association of declared disability with academic performance. DESIGN: Observational study using record-linked data collected between 2002 and 2018. SETTING: UK Medical Education Database is a repository of data relating to training of medical students and doctors. Disability and other data are record-linked. PARTICIPANTS: All students starting at a UK medical school between 2002 and 2018 (n=135 930). MAIN OUTCOME MEASURES: Declared disability was categorised by the Higher Education Statistics Authority. Outcomes related to undergraduate academic performance included scores in the educational performance measure (EPM), prescribing safety assessment and situational judgement test. Performance in postgraduate examinations was studied, as well as prior attainment in school examinations and aptitude tests. RESULTS: Specific learning disability (SLD) was the most commonly declared disability (3.5% compared with the next most commonly declared disability at 1.0% of n=129 345 all cases in the study), and during the period covered by the data, SLD declarations increased from 1.4% (n=6440 for students starting in 2002) to 4.6% (n=8625 for students starting in 2018). In a logistic regression, the following factors predicted recording of SLD on entry to medical school ((exp(B)±95% CI), p<0.0001 unless otherwise stated): attendance at a fee-paying school (2.306±0.178), graduate status (1.806±0.205), participation of local areas quintile (1.089±0.030), age (1.034±0.012). First year medical students were less likely to declare SLD if they were from a non-white ethnic background (Asian/Asian British 0.324±0.034, black/black British 0.571±0.102, mixed 0.731±0.108, other ethnic groups 0.566±0.120), female (0.913±0.059; p=0.007) or from a low index of multiple deprivation quintile (0.963±0.029); p=0.017. In univariate analysis with Bonferroni corrections applied for multiple tests, no significant difference was observed in the recording of SLD according to socioeconomic class (χ2=5.637, p=1), whether or not a student's parents had a higher education (χ2=0.140, p=1), or whether or not a student had received a United Kingdom Clinical Aptitude Test (UKCAT) bursary (χ2=7.661, p=0.068). Students who declared SLD at some point in medical school (n=4830) had lower EPM normalised deviate values (-0.390) than those who did not (-0.119) (F=189.872, p<0.001). Those for whom SLD was recorded were as likely to complete the course successfully as those who did not declare disability (93.0% successful completion by those for whom SLD declared from year 1 (n=2480), 92.2% by those for whom SLD declared after year 1 (n=2350), 91.6% by those for whom SD not declared at any point (n=85 180)) (χ2=6.905, p=0.032). Of 3580 first year students who declared SLD, 43.1% had not sat the UKCAT Special Educational Needs aptitude test (which gives extra time for those with special educational needs), while 28% of 2400 registrants for whom SLD was recorded as medical students did not declare it at General Medical Council registration. CONCLUSIONS: Substantial increases in declaration of SLD may reflect changes in the social and legal environment during the period of the study. Those who declare SLD are just as likely to gain a primary medical qualification as those who do not. For some individuals, disability declaration appears to depend on context, based on differences in numbers declaring SLD before, during and after medical school.

Situational judgement test performance and subsequent misconduct in medical students.

INTRODUCTION: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. METHODS: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. RESULTS: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect-size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. CONCLUSIONS: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection.

UCAT and dental student selection in the UK - what has changed?

Introduction The University Clinical Aptitude Test (UCAT) has been used since 2006 by a consortium of UK medical and dental schools to assist in undergraduate selection. In 2019, UCAT was used by 30 universities (14 dental schools).Aim To report how UCAT use has changed in undergraduate student selection in the UK.Methods UCAT use was categorised and trends identified from annual telephone interviews with dental school admission tutors; this process started in 2011.Results Dental schools using UCAT rose from 8 (2006) to 14 (2020). The most significant use of the test to select applicants for interview was as a weighted factor; at offer stage, UCAT was most used to discriminate between applicants at borderlines. A growing number of dental schools are using the Situational Judgement Test (SJT) in selection (2019, n = 6). In 2019, eight schools adjusted selection processes for widening access applicants. Multiple mini interviews are now used by the majority (n = 10) of dental schools.Conclusions UCAT represents a significant factor in selection to UK undergraduate dental programmes and is used by all but two dental schools. In most schools, UCAT contributes in a substantial way to selection outcomes and strength in test use has grown over time.

Can achievement at medical admission tests predict future performance in postgraduate clinical assessments? A UK-based national cohort study.

OBJECTIVE: To determine whether scores on two undergraduate admissions tests (BioMedical Admissions Test (BMAT) and University Clinical Aptitude Test (UCAT)) predict performance on the postgraduate Membership of the Royal Colleges of Physicians (MRCP) examination, including the clinical examination Practical Assessment of Clinical Examination Skills (PACES). DESIGN: National cohort study. SETTING: Doctors who graduated medical school between 2006 and 2018. PARTICIPANTS: 3045 doctors who had sat BMAT, UCAT and the MRCP. PRIMARY OUTCOME MEASURES: Passing each section of the MRCP at the first attempt, including the clinical assessment PACES. RESULTS: Several BMAT and UCAT subtest scores displayed incremental predictive validity for performance on the first two (written) parts of the MRCP. Only aptitude and skills on BMAT (OR 1.34, 1.08 to 1.67, p=0.01) and verbal reasoning on UCAT (OR 1.34, 1.04 to 1.71, p=0.02) incrementally predicted passing PACES at the first attempt. CONCLUSIONS: Our results imply that the abilities assessed by aptitude and skills and verbal reasoning may be the most important cognitive attributes, of those routinely assessed at selection, for predicting future clinical performance. Selectors may wish to consider placing particular weight on scales assessing these attributes if they wish to select applicants likely to become more competent clinicians. These results are potentially relevant in an international context too, since many admission tests used globally, such as the Medical College Admission Test, assess similar abilities.

An investigation of the predictive validity of selection tools on performance in physiotherapy training in Australia.

OBJECTIVES: Despite a large body of research on selection in medical education, very little is conducted in other health professions. This study investigated the predictive validity of multiple selection tools on academic and clinical performance outcomes of undergraduate physiotherapy students. DESIGN: A retrospective observational study. SETTING: Undergraduate physiotherapy program in Australia. PARTICIPANTS: 497 undergraduate physiotherapy students across seven entry cohorts. Including students directly from secondary school (n=381) and with prior tertiary study (n=116). MAIN OUTCOME MEASURES: Academic performance as measured by written examinations. Clinical performance, measured by Objective Structured Clinical Examinations (OSCEs) during on-campus units and the Assessment of Physiotherapy Practice (APP) for off-campus clinical placements. Predictor variables included selection tools (academic achievement, interview, aptitude test) and demographic variables (age, gender). RESULTS: Selection interview was a positive predictor of OSCEs and final year clinical performance in direct school leaver participants. Academic achievement scores from selection positively predicted written examinations scores. CONCLUSION: Clinical and academic performance were predicted by tools measuring different domains at selection. Assessing broadly across academic and non-academic domains at selection can be valuable in identifying applicants who will be able to meet the range of outcomes for course completion and subsequent registration in the physiotherapy profession.

Wax carving exercises predict performance on practical examinations for first-year dental students.

Perceptual ability test (PAT) is a valid determinant for spatial perceptions. However, a validated psychomotor skill test for dentistry does not currently exist. PURPOSE/OBJECTIVES: This study aimed to determine whether the results of two wax carving exercises (E1 and E2), PAT, quantitative reasoning (QR), and academic average (AA) tests predict students' performance on dental anatomy practical examinations. METHODS: First-year dental students (n = 69) participated in the study. Results of the PAT, QR, and AA were obtained from the Admissions Office. Participants completed wax carving exercises during the first (E1) and last weeks (E2) of the dental anatomy course. Carving instruments, instruction, and rubrics, were provided to participants who were instructed to carve a cube and a semicircle shape into a wax block using the wax subtraction technique. Two calibrated and blinded faculty graded the exercise as ideal, satisfactory, or unsatisfactory in four categories: finish, sharpness, symmetry, and accuracy. During the dental anatomy course, participants completed wax carving practical examinations for a maxillary central incisor and a mandibular first molar. Examinations were graded by calibrated faculty as ideal, satisfactory, or unsatisfactory. The linear mixed effects model assessed influences of independent variables on the practical examinations. RESULTS: estimates of the independent variables on the practical examinations were PAT (0.027), AA (0.088), QR (-0.014), E1 (0.235), and E2 (0.175). CONCLUSIONS: wax carving exercises were stronger predictors of performance on the practical examinations than the PAT, AA, and QR. This simple exercise may help identify students early in their preclinical education requiring additional assistance.