Evaluating a Situational Judgment Test for Use in Medical School Admissions: Two Years of AAMC PREview Exam Administration Data.

PURPOSE: To examine the relationship between the Association of American Medical Colleges (AAMC) Professional Readiness Exam (PREview) scores and other admissions data, group differences in mean PREview scores, and whether adding a new assessment tool affected the volume and composition of applicant pools. METHOD: Data from the 2020 and 2021 PREview exam administrations were analyzed. Two U.S. schools participated in the PREview pilot in 2020 and 6 U.S. schools participated in 2021. PREview scores were paired with data from the American Medical College Application Service (undergraduate grade point averages [GPAs], Medical College Admission Test [MCAT] scores, race, and ethnicity) and participating schools (interview ratings). RESULTS: Data included 19,525 PREview scores from 18,549 unique PREview examinees. Correlations between PREview scores and undergraduate GPAs ( r = .16) and MCAT scores ( r = .29) were small and positive. Correlations between PREview scores and interview ratings were also small and positive, ranging between .09 and .14 after correcting for range restriction. Small group differences in mean PREview scores were observed between White and Black or African American and White and Hispanic, Latino, or of Spanish origin examinees. The addition of the PREview exam did not substantially change the volume or composition of participating schools' applicant pools. CONCLUSIONS: Results suggest the PREview exam measures knowledge of competencies that are distinct from those measured by other measures used in medical school admissions. Observed group differences were smaller than group differences observed with traditional academic assessments and evaluations. The addition of the PREview exam did not substantially change the overall volume of applications or the proportions of out-of-state, underrepresented in medicine, or lower socioeconomic status applicants. While more research is needed, these results suggest the PREview exam may provide unique information to the admissions process without adversely affecting applicant pools.

Creating a Diverse Physician Workforce Through Training in Diversity, Bias, and Selection: An Innovative Approach to Preparing Medical Students as Future Academic Leaders.

Medical students are the future of academic medicine. They will serve as admissions committee members, deans, and program directors with responsibility for selecting future physicians. At the same time, schools and programs are working to diversify the physician workforce to care for diverse patient populations. At UAB Heersink School of Medicine, we developed an elective course, Holistic Review: Creating a Mission-Driven Physician Workforce, to train learners in bias, diversity, and data-informed decision-making. Students applied lessons to the admissions process by participating in application screening and multiple-mini interview rating. Student reflections demonstrated course applicability for careers in medicine and beyond.

The Performance and Trajectory of Medical Students With Disabilities: Results From a Multisite, Multicohort Study.

PURPOSE: To conduct a post-Americans with Disabilities Act Amendments Act of 2008 multisite, multicohort study called the Pathways Project to assess the performance and trajectory of medical students with disabilities (SWDs). METHOD: From June to December 2020, the authors conducted a matched cohort study of SWDs and nondisabled controls from 2 graduating cohorts (2018 and 2019) across 11 U.S. MD-granting medical schools. Each SWD was matched with 2 controls, one from their institution and, whenever possible, one from their cohort for Medical College Admission Test score and self-reported gender. Outcome measures included final attempt Step 1 and Step 2 Clinical Knowledge scores, time to graduation, leave of absence, matching on first attempt, and matching to primary care. RESULTS: A total of 171 SWDs and 341 controls were included; the majority of SWDs had cognitive/learning disabilities (118/171, 69.0%). Compared with controls, SWDs with physical/sensory disabilities had similar times to graduation (88.6%, 95% confidence interval [CI]: 77.0, 100.0 vs 95.1%, 95% CI: 90.3, 99.8; P = .20), Step 1 scores (229.6 vs 233.4; P = .118), and match on first attempt (93.9%, 95% CI: 86.9, 100.0 vs 94.6%, 95% CI: 91.8, 97.4; P = .842), while SWDs with cognitive/learning disabilities had lower Step 1 scores (219.4; P < .001) and were less likely to graduate on time (81.2%, 95% CI: 69.2, 93.2; P = .003) and match on first attempt (85.3%, 95% CI: 78.0, 92.7; P = .009). Accommodated SWDs had Step 1 scores that were 5.9 points higher than nonaccommodated SWDs (95% CI: -0.7, 12.5; P = .08). CONCLUSIONS: Structural barriers remain for SWDs with cognitive/learning disabilities, which could be partially mitigated by accommodations on high-stakes exams.

Enrollment Management in Undergraduate Medical School Admissions: A Complementary Framework to Holistic Review for Increasing Diversity in Medicine.

Medical schools implemented holistic review more than a decade ago, which led to more deliberate consideration and inclusion of applicants historically underrepresented in medicine. This article presents a theory of holistic enrollment management that unites holistic review with enrollment management principles. This theory contextualizes medical school admissions as a complex marketplace with multifaceted, competing forces. Applying an enrollment management framework of mission, market, means, and metrics can improve the capacity of a medical school to efficiently advance its mission over time. Medical schools employing a clear, compelling, and focused mission to direct all aspects of the medical education enterprise can more effectively attract applicants who are better prepared to enact that mission throughout their careers. Medical schools share a marketplace and collectively compete to identify, attract, admit, and matriculate the most mission-aligned student body within the pool of applicants they share. Institutions that deliberately mobilize resources within this dynamic marketplace will engage, admit, and matriculate the most suiting applicants and attract even more mission-aligned matriculants over time. Widespread adoption of this holistic framework of enrollment management may enhance the capacity of the medical education system to better capitalize on the existing diversity in the national pool of applicants, encourage more underrepresented applicants to apply in the future, admit and matriculate a more diverse national student body, and ultimately better prepare new physicians to meet the increasingly diverse health care needs of the nation.

Impact of holistic review on student interview pool diversity.

Diversity in the physician workforce lags behind the rapidly changing US population. Since the gateway to becoming a physician is medical school, diversity must be addressed in the admissions process. The Association of American Medical Colleges has implemented a Holistic Review Initiative aimed at assisting medical schools with broadening admission criteria to include relevant, mission-driven attributes and experiences in addition to academic preparation to identify applicants poised to meet the needs of a diverse patient population. More evidence is needed to determine whether holistic review results in a more diverse selection process. One of the keys to holistic review is to apply holistic principles in all stages of the selection process to ensure qualified applicants are not overlooked. This study examines whether the use of holistic review during application screening at a new medical school increased the diversity of applicants selected for interview. Using retrospective data from the first five application cycles at the Oakland University William Beaumont School of Medicine (OUWB), the author compared demographic and experiential differences between the applicants selected using holistic review, including experiences, attributes and academic metrics, to a test sample selected solely using academic metrics. The dataset consisted of the total group of applicants selected for interview in 2011 through 2015 using holistic review (n = 2773) and the same number of applicants who would have been selected for an interview using an academic-only selection model (n = 2773), which included 1204 applicants who were selected using both methods (final n = 4342). The author used a combination of cross-tabulation and analysis of variance to identify differences between applicants selected using holistic review and applicants in the test sample selected using only academics. The holistic review process yielded a significantly higher than expected percent of female (adj. resid. = 13.2, p < .01), traditionally underrepresented in medicine (adj. resid. = 15.8, p < .01), first generation (adj. resid. = 5.8, p < .01), and self-identified disadvantaged (adj resid. = 11.5, p < .01) applicants in the interview pool than selected using academic metrics alone. In addition, holistically selected applicants averaged significantly more hours than academically selected students in the areas of pre-medical school paid employment (F = 10.99, mean difference = 657.99, p < .01) and community service (F = 15.36, mean difference = 475.58, p < .01). Using mission-driven, holistic admissions criteria comprised of applicant attributes and experiences in addition to academic metrics resulted in a more diverse interview pool than using academic metrics alone. These findings add support for the use of holistic review in the application screening process as a means for increasing diversity in medical school interview pools.