The added value of free preparatory activities for widening access to medical education: a multi-cohort study.

BACKGROUND: Medical schools are reported to be less accessible to students with non-traditional backgrounds. These students face barriers when applying for and transitioning to medical school, which may be reduced by offering free preparatory activities. By equalizing access to resources, these activities are expected to reduce disparities in selection outcomes and early academic performance. In the present study, four free institutionally-provided preparatory activities were evaluated by comparing the demographic composition of participating and non-participating applicants. Additionally, the association between participation and selection outcomes and early academic performance was investigated for subgroups (based on sex, migration background and parental education). METHODS: Participants were applicants to a Dutch medical school in 2016-2019 (N = 3592). Free preparatory activities included Summer School (N = 595), Coaching Day (N = 1794), Pre-Academic Program (N = 217), and Junior Med School (N = 81), supplemented with data on participation in commercial coaching (N = 65). Demographic compositions of participants and non-participants were compared using chi-squared tests. Regression analyses were performed to compare selection outcomes (curriculum vitae [CV], selection test score, probability of enrolment) and early academic performance (first-course grade) between participants and non-participants of demographic subgroups, controlling for pre-university grades and participation in other activities. RESULTS: Generally, no differences in sociodemographic compositions of participants and non-participants were found, but males participated less often in Summer School and Coaching Day. Applicants with a non-Western background participated less often in commercial coaching, but the overall participation rate was low and participation had negligible effects on selection outcomes. Participation in Summer School and Coaching Day were stronger related with selection outcomes. In some cases, this association was even stronger for males and candidates with a migration background. After controlling for pre-university grades, none of the preparatory activities were positively associated with early academic performance. CONCLUSIONS: Free institutionally-provided preparatory activities may contribute to student diversity in medical education, because usage was similar across sociodemographic subgroups, and participation was positively associated with selection outcomes of underrepresented and non-traditional students. However, since participation was not associated with early academic performance, adjustments to activities and/or curricula are needed to ensure inclusion and retention after selection.

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Situational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used.