Integrity situational judgement test for medical school selection: judging 'what to do' versus 'what not to do'.

CONTEXT: Despite their widespread use in medical school selection, there remains a lack of clarity on exactly what situational judgement tests (SJTs) measure. OBJECTIVES: We aimed to develop an SJT that measures integrity by combining critical incident interviews (inductive approach) with an innovative deductive approach. The deductive approach guided the development of the SJT according to two established theoretical models, of which one was positively related to integrity (honesty-humility [HH]) and one was negatively related to integrity (cognitive distortions [CD]). The Integrity SJT covered desirable (HH-based) and undesirable (CD-based) response options. We examined the convergent and discriminant validity of the Integrity SJT and compared the validity of the HH-based and CD-based subscores. METHODS: The Integrity SJT was administered to 402 prospective applicants at a Dutch medical school. The Integrity SJT consisted of 57 scenarios, each followed by four response options, of which two represented HH facets and two represented CD categories. Three SJT scores were computed, including a total, an HH-based and a CD-based score. The validity of these scores was examined according to their relationships with external integrity-related measures (convergent validity) and self-efficacy (discriminant validity). RESULTS: The three SJT scores correlated significantly with all integrity-related measures and not with self-efficacy, indicating convergent and discriminant validity. In addition, the CD-based SJT score correlated significantly more strongly than the HH-based SJT score with two of the four integrity-related measures. CONCLUSIONS: An SJT that assesses the ability to correctly recognise CD-based response options as inappropriate (i.e. what one should not do) seems to have stronger convergent validity than an SJT that assesses the ability to correctly recognise HH-based response options as appropriate (i.e. what one should do). This finding might be explained by the larger consensus on what is considered inappropriate than on what is considered appropriate in a challenging situation. It may be promising to focus an SJT on the ability to recognise what one should not do.

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection.

BACKGROUND: This study examined the influence of two Situational Judgement Test (SJT) design features (response instructions and response format) on applicant perceptions. Additionally, we investigated demographic subgroup differences in applicant perceptions of an SJT. METHODS: Medical school applicants (N = 372) responded to an online survey on applicant perceptions, including a description and two example items of an SJT. Respondents randomly received one of four SJT versions (should do-rating, should do-pick-one, would do-rating, would do-pick-one). They rated overall favourability and items on four procedural justice factors (face validity, applicant differentiation, study relatedness and chance to perform) and ease-of-cheating. Additionally, applicant perceptions were compared for subgroups based on gender, ethnic background and first-generation university status. RESULTS: Applicants rated would-do instructions as easier to cheat than should-do instructions. Rating formats received more favourable judgements than pick-one formats on applicant differentiation, study-relatedness, chance to perform and ease of cheating. No significant main effect for demographic subgroup on applicant perceptions was found, but significant interaction effects showed that certain subgroups might have more pronounced preferences for certain SJT design features. Specifically, ethnic minority applicants - but not ethnic majority applicants - showed greater preference for should-do than would-do instructions. Additionally, first-generation university students - but not non-first-generation university students - were more favourable of rating formats than of pick-one formats. CONCLUSIONS: Findings indicate that changing SJT design features may positively affect applicant perceptions by promoting procedural justice factors and reducing perceived ease of cheating and that response instructions and response format can increase the attractiveness of SJTs for minority applicants.

Participation in a scientific pre-university program and medical students' interest in an academic career.

BACKGROUND: The proportion of medical doctors involved in research activities is declining. Undergraduate medical research programs are positively associated with medical students' research interest. Scientific pre-university programs (SPUPs) outside the medical domain are also positively associated with research interest, but have not been related to the shortage of clinician-scientists. This study examined the effect of an SPUP on medical students' research interest. METHODS: This study was conducted at a Dutch medical school. Medical students in all years who had participated in an SPUP and first-year master students who had not participated in an SPUP were invited to fill out an online survey on extracurricular activities and future career interests. SPUP participants were compared with three groups of non-participants: (i) an unmatched group, (ii) a group matched on gender and pre-university Grade Point Average (pu-GPA) and (iii) a group matched on gender and first-year GPA, one to five years after finishing the SPUP. Participants evaluated the SPUP through ratings of statements about the program. RESULTS: Two-hundred forty medical students, including 71 SPUP participants responded to the survey. SPUP participants participated significantly more often in the Honors class (i.e., extracurricular educational program for high-performing students), gained significantly more often extracurricular research experience, enrolled significantly more often in the Research master (i.e., research training program parallel to the clinical master program) and obtained significantly more often a scholarship than unmatched non-SPUP participants. Using a non-SPUP group matched on gender and pu-GPA reduced the effect size of the significant differences in Honors class participation, Research master participation and scholarship obtainment. Using a non-SPUP group matched on gender and first-year GPA rendered the significant difference in Research master participation and scholarship obtainment insignificant. Significantly more SPUP participants than unmatched non-SPUP participants preferred a combination of clinical care and research in their future career. Using a non-SPUP group matched on gender and either pu-GPA or first-year GPA did not change the effect size of this significant difference. CONCLUSIONS: These findings demonstrate the potential value of an SPUP in increasing the number of medical students with research interest and as a policy measure to help to alleviate the shortage of clinician-scientists.