Assessing the predictive validity of pre-admission criteria on dental students' academic performance: a cross-sectional study.

BACKGROUND: This study examines the multifaceted factors influencing academic performance among dental students in Saudi Arabia, exploring the complex relationships between demographic, educational, and psychological variables and students' Grade Point Averages (GPAs) to enhance global dental education admission strategies. METHODS: Data on demographics, academic performance indicators (including GPA, high school grades, test scores), accommodation type, parental education, suspension history, and English proficiency were collected through an English-language electronic questionnaire from 900 dental students across multiple institutions in Riyadh. The study employed Pearson's correlation analysis to examine relationships between GPA and various academic and demographic factors. A detailed regression analysis, using a stepwise selection based on Akaike's information criteria (AIC), identified significant GPA predictors and evaluated the average marginal effects of pre-admission variables on GPA, alongside rigorous diagnostic checks to validate the model's robustness. RESULTS: The study revealed a complex interplay of factors impacting GPA among dental students. High school grades, achievement, and aptitude test scores showed moderate positive correlations with GPA, while negative correlations were observed with age and number of times suspended. Regression analysis highlighted age, marital status, high school grades, and test scores as key predictors, with complex interaction effects demonstrating the layered influences of these factors. For instance, the negative impact of age on GPA was moderated by high school grade and aptitude test score. The analysis also highlighted the significant marginal effects of various pre-admission variables on GPA, such as the negative average impact of age and the positive impact of high school grades. CONCLUSIONS: This study provides valuable insights into the multifaceted determinants of academic success in dental education. Our findings underscore the significant roles of high school grades, achievement, and aptitude test scores, as well as the nuanced influence of age and marital status on GPA. These results advocate for a holistic approach in evaluating candidates for dental programs and have broader implications for global dental education, emphasizing the need for comprehensive admission strategies.

Effectiveness of a manual test at the beginning of dental school.

INTRODUCTION: This study analysed whether the SAZAMED (Self-assessment for dentistry studies) manual test conducted as an exercise or examination in the first year of dental studies, and the official aptitude test for medical studies (EMS), can predict future performance of dental students. MATERIALS AND METHODS: Students enrolled in the dental curriculum at the University of Basel between 2009/10 and 2019/20 were categorized: Group A conducted the SAZAMED in the third bachelor year (BA) and second master year (MA), group B practiced in the first and third BA and second MA and group C had to pass the test in the first BA with repetition in the third BA and second MA. SAZAMED comprised (i) wire bending, (ii) modelling sphere and cube, (iii) drawing mirror-inverted and (iv) indirect drawing over a mirror. Comparative variables were the EMS per cent rank and test scores in practical-clinical examinations in the third BA and the second MA. RESULTS: Data were available from 329 students over the 10-year period. Repeaters and dropouts had poorer performance in the first BA SAZAMED than students who did not (p < 0.001). Among the four SAZAMED exercises, mirror-inverted drawing was the strongest predictor of future student performance. The official aptitude test EMS correlated with the third BA certificate (p = 0.012), but failed to predict repeaters and dropouts. CONCLUSION: Since academic success was associated with significantly better scores on the first BA SAZAMED, it is considered a predictor and recommended in addition to the official EMS to identify students in need of intervention programmes or who should choose a different field of study.

Do students' personality traits change during medical training? A longitudinal cohort study.

Many medical schools incorporate assessments of personal characteristics, including personality traits, in their selection process. However, little is known about whether changes in personality traits during medical training affect the predictive validity of personality assessments. The present study addressed this issue by examining the stability of personality traits and their predictive validity over a 6-year medical training course. Participants were two cohorts of Swiss medical students (N = 272, 72% of students admitted to Year 2) from whom we collected demographic data, Swiss medical studies aptitude test (EMS) scores, Big Five personality traits scores measured at three times and scores on the multiple-choice and objective structured clinical examination parts of the final medical examination. Our findings indicated that personality traits had medium-to-high rank-order stability (r > .60 over 3 years and r > .50 over 6 years). Mean-level changes were moderate for agreeableness (d = + 0.72) and small for neuroticism and conscientiousness (d = -0.29, d = -0.25, respectively). Individual reliable change indices ranged from 4.5% for openness to 23.8% for neuroticism. The predictive validity was similar to that of the first three years of follow-up. To the best of our knowledge, this is the first study to investigate changes in personality across undergraduate curriculum. Medical students' personality traits were mostly stable across medical school and retain their predictive validity. Consequently, this study supports the use of tools measuring constructs underlying personality traits in selection. In addition, this study confirms that examination formats could favor students with certain personality traits.

Longitudinally adaptive assessment and instruction increase numerical skills of preschool children.

Social inequality in mathematical skill is apparent at kindergarten entry and persists during elementary school. To level the playing field, we trained teachers to assess children's numerical and spatial skills every 10 wk. Each assessment provided teachers with information about a child's growth trajectory on each skill, information designed to help them evaluate their students' progress, reflect on past instruction, and strategize for the next phase of instruction. A key constraint is that teachers have limited time to assess individual students. To maximize the information provided by an assessment, we adapted the difficulty of each assessment based on each child's age and accumulated evidence about the child's skills. Children in classrooms of 24 trained teachers scored 0.29 SD higher on numerical skills at posttest than children in 25 randomly assigned control classrooms (P = 0.005). We observed no effect on spatial skills. The intervention also positively influenced children's verbal comprehension skills (0.28 SD higher at posttest, P < 0.001), but did not affect their print-literacy skills. We consider the potential contribution of this approach, in combination with similar regimes of assessment and instruction in elementary schools, to the reduction of social inequality in numerical skill and discuss possible explanations for the absence of an effect on spatial skills.

BMAT's predictive validity for medical school performance: A retrospective cohort study.

BACKGROUND: Although used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools. METHODS: This was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students. RESULTS: When BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades. DISCUSSION: Schools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Situational judgement test performance and subsequent misconduct in medical students.

INTRODUCTION: Situational judgement tests (SJTs) have been widely adopted, internationally, into medical selection. It was hoped that such assessments could identify candidates likely to exhibit future professional behaviours. Understanding how performance on such tests may predict the risk of disciplinary action during medical school would provide evidence for the validity of such SJTs within student selection. It would also inform the implementation of such tests within student recruitment. METHODS: This cohort study used data for 6910 medical students from 36 UK medical schools who sat the University Clinical Aptitude Test (UCAT) SJT in 2013. The relationship between SJT scores at application and the risk of subsequent disciplinary action during their studies was modelled. The incremental ability of the SJT scores to predict the risk of disciplinary action, above that already provided by UCAT cognitive test scores and secondary (high) school achievement, was also evaluated in 5535 of the students with information available on this latter metric. RESULTS: Two hundred and ten (3.05%) of the students in the cohort experienced disciplinary action. The risk of disciplinary action reduced with increasing performance on the admissions SJT (odds ratio (OR) 0.80, 95% confidence interval (CI) 0.69 to 0.92, p = 0.002). This effect remained similar after adjusting for cognitive performance and prior academic attainment (OR 0.77, 95% CI 0.65 to 0.92, p = 0.004). The overall estimated effect-size was small (Cohen's d = 0.08) and no evidence of 'threshold' effects were observed for the SJT scores and risk of disciplinary action. CONCLUSIONS: Performance on admissions SJTs can, at least modestly, incrementally predict the risk of subsequent disciplinary action, supporting their use in this context. However, for this SJT and outcome, there did not seem a distinct threshold score above which the risk of disciplinary action disproportionately increased. This should be considered when using the scores within medical selection.

Performance at medical school selection correlates with success in Part A of the intercollegiate Membership of the Royal College of Surgeons (MRCS) examination.

Medical schools in the UK typically use prior academic attainment and an admissions test (University Clinical Aptitude Test (UCAT), Biomedical Admissions Test (BMAT) or the Graduate Medical School Admissions Test (GAMSAT)) to help select applicants for interview. To justify their use, more information is needed about the predictive validity of these tests. Thus, we investigated the relationship between performance in admissions tests and the Membership of the Royal College of Surgeons (MRCS) examination.The UKMED database (https://www.ukmed.ac.uk) was used to access medical school selection data for all UK graduates who attempted MRCS Part A (n=11 570) and Part B (n=5690) between 2007 and 2019. Univariate and multivariate logistic regression models identified independent predictors of MRCS success. Pearson correlation coefficients examined the linear relationship between test scores and MRCS performance.Successful MRCS Part A candidates scored higher in A-Levels, UCAT, BMAT and GAMSAT (p<0.05). No significant differences were observed for MRCS Part B. All admissions tests were found to independently predict MRCS Part A performance after adjusting for prior academic attainment (A-Level performance) (p<0.05). Admission test scores demonstrated statistically significant correlations with MRCS Part A performance (p<0.001).The utility of admissions tests is clear with respect to helping medical schools select from large numbers of applicants for a limited number of places. Additionally, these tests appear to offer incremental value above A-Level performance alone. We expect this data to guide medical schools' use of admissions test scores in their selection process.

Assessing the predictive validity of the UCAT-A systematic review and narrative synthesis.

BACKGROUND: The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors and dentists. We conducted a systematic review to establish the predictive value of the UCAT across measures of performance at undergraduate and post-graduate levels. METHODS: A literature search was conducted in April 2020 using eight electronic databases: MEDLINE, APA PsycInfo, SCOPUS, Web of Science, EThOS, OpenGrey, PROSPERO, and the UCAT website. Data were extracted from selected studies and tabulated as results matrices. A narrative synthesis was performed. RESULTS: Twenty-four studies satisfied our inclusion criteria, 23 of which were deemed to be of good quality (using the Newcastle-Ottawa Scale). For over 70% of univariate data points, the UCAT exerted no statistically significant predictive validity; for the remainder, predictive power was weak. The cognitive total and verbal reasoning subtests had the largest evidence base as weakly positive predictors of academic performance. The SJT subtest was a weak predictor of professional behaviour during medical school. Studies specific to dental schools demonstrated variable findings across the five studies. Only 1 study looked at post-graduate outcome measures and demonstrated that the UCAT was not a predictor of health- or conduct-related fitness to practice declarations at GMC registration. CONCLUSIONS: These data provide some support for the use of cognitive total and verbal reasoning subtests as part of medical school selection. Further research is needed to investigate outcomes beyond professional registration and for dental students.

An assessment of the impact of formal preparation activities on performance in the University Clinical Aptitude Test (UCAT): a national study.

BACKGROUND: Previous studies have shown performance in the University Clinical Aptitude Test (UCAT) to be associated with measures of candidate socio-economic advantage such as parental occupation and type of school attended. It is possible that access to preparation support and materials may in part explain these associations. In this paper we determine whether use of formal preparation resources is associated with higher UCAT scores and whether differences in use of preparation resources exist between socio-demographic groups. METHODS: After completing the 2017 UCAT UK school-leaver candidates (n = 14,332) were asked to answer a questionnaire regarding their use of official UCAT and commercial resources, school-based support, and time spent preparing. Multiple linear and logistic regression models were used to evaluate the associations between preparedness, demographic characteristics and UCAT performance. RESULTS: Five thousand, four hundred thirty-nine (38%) candidates responded to the questionnaire. Use of freely available UCAT official practice tests, paid commercial materials, attendance at school-based preparation courses and spending more time preparing were significantly associated with higher UCAT scores. Candidates who were from less deprived backgrounds and attending independent or grammar schools were significantly more likely to use paid commercial materials and spend longer preparing. CONCLUSIONS: Reported use of preparation resources varies between candidates from different socio-demographic backgrounds and is associated independently with performance in the UCAT. Increasing the availability of freely available resources may mitigate some of these differences.

The change from UMAT to UCAT for undergraduate medical school applicants: impact on selection outcomes.

OBJECTIVES: To assess whether the change from the Undergraduate Medical and Health Sciences Admissions Test (UMAT; 1991-2019) to the University Clinical Aptitude Test (UCAT) for the 2020 New South Wales undergraduate medical degree intake was associated with changes in the impact of sex, socio-economic status and remoteness of residence, and professional coaching upon selection for interview. DESIGN, SETTING, PARTICIPANTS: Cross-sectional study of applicants for the three NSW undergraduate medical programs for entry in 2019 (4114 applicants) or 2020 (4270); 703 people applied for both intakes. Applicants selected for interview were surveyed about whether they had received professional coaching for the selection test. MAIN OUTCOME MEASURES: Scores on the three sections of the UMAT (2019 entry cohort) and the five subtests of the UCAT (2020 entry); total UMAT and UCAT scores. RESULTS: Mean scores for UMAT 1 and 3 and for all four UCAT cognitive subtests were higher for men than women; the differences were statistically significant after adjusting for age, socio-economic status, and remoteness. The effect size for sex was 0.24 (95% CI, 0.18-0.30) for UMAT total score, 0.38 (95% CI, 0.32-0.44) for UCAT total score. For the 2020 intake, 2303 of 4270 applicants (53.9%) and 476 of 1074 interviewees (44.3%) were women. The effect size for socio-economic status was 0.47 (95% CI, 0.39-0.54) for UMAT, 0.43 (95% CI, 0.35-0.50) for UCAT total score; the effect size for remoteness was 0.54 (95% CI, 0.45-0.63) for UMAT, 0.48 (95% CI, 0.39-0.58) for UCAT total score. The impact of professional coaching on UCAT performance was not statistically significant among those accepted for interview. CONCLUSIONS: Women and people from areas outside major cities or of lower socio-economic status perform less well on the UCAT than other applicants. Reviewing the test and applicant quotas may be needed to achieve selection equity.

Relationship between admission criteria and academic performance in basic science courses in health science colleges in KAU.

BACKGROUND: At King Abdulaziz University, medical and health science schools depend on admission exams (aptitude and achievement) and preparatory year scores in their students' selection. However, with the growing number of applicants and the drastic changes in teaching and assessment in these colleges, continuous assessment and development of admission criteria are needed. In this study, we aimed to evaluate the correlation of admission exam scores, in addition to the preparatory year Grade Point Average (GPA), with academic performance in the basic science subjects such as Clinical Biochemistry and Clinical Pharmacology in health science colleges. METHODS: The study was conducted on four cohort studies, two faculty of nursing cohorts; nursing students (2017-2018, n=146) nursing students (2018-2019, n=81), and two faculty of applied medical sciences cohorts, clinical nutrition students (2017-2018, n=33), and clinical nutrition students (2018-2019, n=28). The students' scores of General Aptitude Test (GAT), Scholastic Achievement Admission Test (SAAT), and preparatory year GPA were all recorded at the beginning of each semester before the beginning of courses. Clinical Biochemistry and Clinical Pharmacology exam results were recorded at the end of the semester. Correlation was done for each cohort and all cohorts pooled. RESULTS: Results showed only a weak correlation detected between SAAT and the overall achievement in Clinical Biochemistry (r= 0.192, P= 0.042) in nursing students (2017-2018), but no correlation was seen with SAAT or preparatory year scores. There was also no significant correlation between admission exams scores and the students' academic achievement in Clinical Biochemistry or Clinical Pharmacology. On the other hand Clinical Pharmacology exam results showed a significant positive correlation with Clinical Biochemistry results (r=0.688, P=0.000). CONCLUSION: Our results could indicate the need to revisit the admission criteria for these colleges. Furthermore, specific preparatory year tracks for health science colleges can ensure that students improve the specific skills and knowledge required for their future college years3.

Emotional Intelligence Predicts Accurate Self-Assessment of Surgical Quality: A Pilot Study.

BACKGROUND: Self-assessment is fundamental in surgical training. Accuracy of self-assessment is superior with greater age, experience, and the use of video playback. Presently, there is scarce evidence in the literature regarding predictors for a surgical trainee's aptitude for self-assessment. The objective of this study was to investigate whether emotional intelligence or visual-spatial aptitude can predict effective self-assessment among novice surgeons performing laparoscopic appendectomy (LA). MATERIALS AND METHODS: Eighteen novice trainees performed a simulated LA, and two aptitude measures were evaluated: (1) emotional intelligence questionnaire and (2) visual spatial ability test. Self-assessment of their performance was conducted using the Objective Assessment of Surgical and Technical Skills global rating scale and ranking five subtasks of the procedure in order of quality of performance after watching a playback of their LA. Two blinded experts (senior consultant surgeons, performed >100 LAs) assessed surgical quality using the same scoring system. Candidates were ranked into higher and lower aptitude groups for the two aptitude measures. Spearman's rank correlation coefficient was calculated to identify if either of the two groups demonstrated greater agreement between self and expert assessment in relation to the two aptitude measures. RESULTS: Participants with a higher degree of emotional intelligence demonstrated significant agreement with expert assessment (r = 0.73, P = 0.031). CONCLUSIONS: Emotional intelligence can predict better self-assessment of surgical quality after performing a simulated LA. This may facilitate early identification of individuals who might require mentoring or guidance with self-assessment as well as contribute to selection criteria.

The economic costs of selecting medical students: An Australian case study.

CONTEXT: The design of selection methods must balance, amongst a range of factors, the desire to select the best possible future doctors with the reality of our resource-constrained environment. Examining the cost of selection processes enables us to identify areas in which efficiencies may be gained. METHODS: A cost description study was conducted based on selection for 2018 entry into medical school directly from secondary school. The perspectives of applicants, volunteer interviewers and the admitting institution were considered. Costs were modelled based on the Monash University (Australia) selection process, which uses a combination of secondary school matriculation score, aptitude test score (Undergraduate Medicine and Health Sciences Admission Test) and multiple mini-interview score. A variety of data sources were utilised, including bespoke surveys, audit data and existing literature. All costs are expressed in 2018 Australian dollars (AU$). Applicant behaviours in preparing for selection tests were also evaluated. RESULTS: A total of 381 of 383 applicants returned the survey. Over 70% of applicants had utilised commercial preparation materials. The median total cost to applicants was AU$2586 (interquartile range [IQR] AU$1574-3999), including costs to both prepare for and attend selection tests. Of 217 volunteer interviewers, 108 returned the survey. These were primarily health professional clinicians at a mid-career stage. The median total cost to interviewers was AU$452 (IQR AU$252-715) for participation in a half-day interview session, largely due to the loss of income. The cost to the admitting institution was AU$269 per applicant, accounted for by the costs of equipment and consumables (52%), personnel (34%) and facilities (14%). CONCLUSIONS: The costs of student selection for medical school are substantial. Understanding costs facilitates achievement of the objective of selecting the desired future medical workforce within the constraints of the resources available. Opportunities for change may arise from changes in applicant preparation behaviours, opportunities for economies of scale, and efficiencies driven by technological solutions.

Opening the black box of selection.

Medical school selection is currently in the paradoxical situation in which selection tools may predict study outcomes, but which constructs are actually doing the predicting is unknown (the 'black box of selection'). Therefore, our research focused on those constructs, answering the question: do the internal structures of the tests in an outcome-based selection procedure reflect the content that was intended to be measured? Downing's validity framework was applied to organize evidence for construct validity, focusing on evidence related to content and internal structure. The applied selection procedure was a multi-tool, CanMEDS-based procedure comprised of a video-based situational judgement test (focused on (inter)personal competencies), and a written aptitude test (reflecting a broader array of CanMEDS competencies). First, we examined content-related evidence pertaining to the creation and application of the competency-based selection blueprint and found that the set-up of the selection procedure was a robust, transparent and replicable process. Second, the internal structure of the selection tests was investigated by connecting applicants' performance on the selection tests to the predetermined blueprint using cognitive diagnostic modeling. The data indicate 89% overlap between the expected and measured constructs. Our results support the notion that the focus placed on creating the right content and following a competency-blueprint was effective in terms of internal structure: most items measured what they were intended to measure. This way of linking a predetermined blueprint to the applicants' results sheds light into the 'black box of selection' and can be used to support the construct validity of selection procedures.

Utility-value intervention with parents increases students' STEM preparation and career pursuit.

During high school, developing competence in science, technology, engineering, and mathematics (STEM) is critically important as preparation to pursue STEM careers, yet students in the United States lag behind other countries, ranking 35th in mathematics and 27th in science achievement internationally. Given the importance of STEM careers as drivers of modern economies, this deficiency in preparation for STEM careers threatens the United States' continued economic progress. In the present study, we evaluated the long-term effects of a theory-based intervention designed to help parents convey the importance of mathematics and science courses to their high-school-aged children. A prior report on this intervention showed that it promoted STEM course-taking in high school; in the current follow-up study, we found that the intervention improved mathematics and science standardized test scores on a college preparatory examination (ACT) for adolescents by 12 percentile points. Greater high-school STEM preparation (STEM course-taking and ACT scores) was associated with increased STEM career pursuit (i.e., STEM career interest, the number of college STEM courses, and students' attitudes toward STEM) 5 y after the intervention. These results suggest that the intervention can affect STEM career pursuit indirectly by increasing high-school STEM preparation. This finding underscores the importance of targeting high-school STEM preparation to increase STEM career pursuit. Overall, these findings demonstrate that a motivational intervention with parents can have important effects on STEM preparation in high school, as well as downstream effects on STEM career pursuit 5 y later.

UKCAT and medical student selection in the UK - what has changed since 2006?

BACKGROUND: The United Kingdom Clinical Aptitude Test (UKCAT) is an aptitude test used since 2006 within selection processes of a consortium of UK medical and dental schools. Since 2006, student numbers have increased in medical training and schools now have an increased focus on widening access. A growing evidence base has emerged around medical student selection (Patterson et al., Med Educ 50:36-60, 2016) leading to changes in practice. However, whilst some papers describe local selection processes, there has been no overview of trends in selection processes over time across Universities. This study reports on how the use of the UKCAT in medical student selection has changed and comments on other changes in selection processes. METHODS: Telephone interviews were conducted annually with UKCAT Consortium medical schools. Use of the UKCAT was categorised and data analysed to identify trends over time. RESULTS: The number of schools using the UKCAT to select applicants for interview has risen, with cognitive test results contributing significantly to outcomes at this stage at many universities. Where schools use different weighted criteria (Factor Method), the UKCAT has largely replaced the use of personal statements. Use of the test at offer stage has also increased; the most significant use being to discriminate between applicants at a decision borderline. A growing number of schools are using the UKCAT Situational Judgement Test (SJT) in selection. In 2018, all but seven (out of 26) schools made some adjustment to selection processes for widening access applicants. Multiple Mini Interviews (MMIs) are now used by the majority of schools. Whilst medical student numbers have increased over this time, the ratio of applicants to places has fallen. The probability of applicants being invited to interview or receiving an offer has increased. CONCLUSIONS: More medical schools are using the UKCAT in undergraduate selection processes in an increasing number of ways and with increasing weight compared with 2007. It has replaced the use of personal statements in all but a few Consortium medical schools. An increased focus on academic attainment and the UKCAT across medical schools may be leading to the need for schools to interview and make offers to more applicants.

Selecting top candidates for medical school selection interviews- a non-compensatory approach.

BACKGROUND: Medical schools apply a range of selection methods to ensure that admitted students succeed in the program. In Australia, selection tools typically include measures of academic achievement (e.g. the Australian Tertiary Admission Rank - ATAR) and aptitude tests (e.g. the Undergraduate Medicine and Health Sciences Admissions Test - UMAT). These are most commonly used to determine which applicants are invited for additional selection processes, such as interviews. However, no previous study has examined the efficacy of the first part of the selection process. In particular, are compensatory or non-compensatory approaches more effective in evaluating the outcomes of cognitive and aptitude tests, and do they affect the demographics of students selected for interview? METHODS: This study utilised data from consecutive cohorts of mainstream domestic students who applied to enter the UNSW Medicine program between 2013 to 2018. A compensatory ranked selection model was compared with a non-compensatory ranked model. Initially, ATAR marks and UMAT scores for each applicant were ranked within each cohort. In the compensatory model, the mean of the ATAR and UMAT ranks were used to determine the outcome. In the non-compensatory model, the lowest rank of ATAR and UMAT determined the outcome for each applicant. The impact of each model on the gender and socioeconomic status of applicants selected to interview was evaluated across all cohorts. RESULTS: The non-compensatory ranked selection model resulted in substantially higher ATAR and UMAT thresholds for invitation to interview, with no significant effect on the socioeconomic status of the selected applicants. CONCLUSIONS: These results are important, demonstrating that it is possible to raise the academic threshold for selection to medicine without having any negative impact on applicants from low socioeconomic backgrounds. Overall, the evidence gathered in this study suggests that a non-compensatory model is preferable for selecting applicants for medical student selection interview.

Many-facet Dichotomous Rasch Model Analysis of the Modern Language Aptitude Test.

Researchers and practitioners have used the Modern Language Aptitude Test (MLAT) to assess language aptitude and identify possible language learning deficiencies in examinees since the 1950s. However, researchers have not assessed its psychometric properties using modern measurement theory methods. We use the dichotomous Rasch model to explore the psychometric properties of the MLAT, including data-model fit indices, item difficulty and student ability calibrations, reliability of separation, and differences in achievement across gender subgroups based on a sample of undergraduate and graduate university students (N=204). Our findings suggest that the MLAT has acceptable psychometric properties such that it can be meaningfully interpreted as a measure of language proficiency. Our findings confirm previous research that language performance across gender groups significantly differs. We found no significant interactions between gender subgroups and the difficulty of the five domains of the assessment. We discuss these results in terms of their implications for research and practice.

Measuring early childhood development in multiple contexts: the internal factor structure and reliability of the early Human Capability Index in seven low and middle income countries.

BACKGROUND: The fourth year of the Sustainable Development Agenda era calls for countries to continue to invest not only in interventions and policies that will promote global equity and sustainability, but also in the monitoring systems required to track progress against these targets. A more pragmatic solution to measuring children's early development in low and middle income countries in particular, is required. This study explores the psychometric properties of the early Human Capability Index (eHCI), a population measure of holistic development for children aged 3-5 years, designed with the vision of being flexible and feasible for use in low resource and capacity settings. METHODS: Utilizing data from seven low and middle income countries: Brazil (n = 1810), China (n = 11,421), Kiribati (n = 8339), Lao People's Democratic Republic (n = 7493), Samoa (n = 12,191), Tonga (n = 6214), and Tuvalu (n = 549), analyses explored the internal factor structure and reliability of scores produced by the tool within each country. RESULTS: Confirmatory factor analyses and internal consistency coefficients demonstrated that after local adaptation, translation, and different implementation methods across countries, the eHCI maintained the same factor structure of nine theoretically-based developmental domains: Physical Health, Verbal Communication, Cultural Knowledge, Social and Emotional Skills, Perseverance, Approaches to Learning, Numeracy, Reading, and Writing. CONCLUSIONS: Findings support the aims of the eHCI in being adaptable and applicable for use within a range of low and middle income countries to facilitate measurement and monitoring of children's early development, as is required for the tracking of progress towards the Sustainable Development Agenda.

Six-minute screening test can provide valid information about the skills that underlie childhood reading and cognitive abilities.

AIM: Reading difficulties can have complex origins and diagnostic measures can be expensive and time consuming. We tested Zippy6, a six-minute screening test that we developed, to see how it compared with a battery of established screening tools. METHODS: This study was carried out at Cincinnati Children's Hospital Medical Center, USA, from 2014-2016. We recruited 81 children with reading difficulties and 44 typical readers of six years to 14 years of age (grades 1-8) using advertisements that were placed in healthcare clinics in the community and on hospital announcement boards. The subjects completed a full reading evaluation, which comprised standardised tests and the new Zippy6 screening test. Standard scores from the traditional and Zippy6 tests were compared, and Support Vector Machine learning methods and algorithms were used to examine the results. RESULTS: Significant positive correlations existed between all of the traditional tests and the Zippy6 screening test. The sensitivity and specificity for the ability of the Zippy6 to discriminate between children with reading difficulties and typical readers were both 75%. CONCLUSION: Using the Zippy6 provided valid information about the skills that underlie reading and cognitive abilities. This screening test could be used by clinicians as either a prediagnostic tool or for postintervention assessment.