Predicting Medical School Enrollment Behavior: Comparing an Enrollment Management Model to Expert Human Judgment.

PURPOSE: Medical school admissions committees are tasked with fulfilling the values of their institutions through careful recruitment. Making accurate predictions regarding enrollment behavior of admitted students is critical to intentionally formulating class composition and impacts long-term physician representation. The predictive accuracy and potential advantages of employing an enrollment predictive model in medical school admissions compared with expert human judgment have not been tested. METHOD: The enrollment management-based predictive model previously generated using historical data was employed to provide a predicted enrollment percentage for each admitted student in the 2016-2017 application pool (N = 352). Concurrently, the human expert created a predicted enrollment percentage for each applicant while blinded to the values generated by the model. An absolute error for each applicant for both approaches was calculated. Statistical significance between approaches (expert vs. enrollment model) was assessed using t tests. RESULTS: The enrollment management approach was noninferior to expert prediction in all cases (P < .05) with a superior correct classification rate (77.7% vs. 71.2%). When considering subgroup analyses for specific populations of potential importance in recruiting (underrepresented in medicine, female, and in-state applicants), the enrollment management predictions were statistically more accurate (P < .05). CONCLUSIONS: Examining a single admitted class, the enrollment predictions using the enrollment management model were at least as accurate as the expert human estimates, and in specific populations of interest more accurate. This information can be readily exported for a real-time dashboard system to drive recruitment behaviors.

Enrollment Management in Medical School Admissions: A Novel Evidence-Based Approach at One Institution.

PURPOSE: In higher education, enrollment management has been developed to accurately predict the likelihood of enrollment of admitted students. This allows evidence to dictate numbers of interviews scheduled, offers of admission, and financial aid package distribution. The applicability of enrollment management techniques for use in medical education was tested through creation of a predictive enrollment model at the University of Michigan Medical School (U-M). METHOD: U-M and American Medical College Application Service data (2006-2014) were combined to create a database including applicant demographics, academic application scores, institutional financial aid offer, and choice of school attended. Binomial logistic regression and multinomial logistic regression models were estimated in order to study factors related to enrollment at the local institution versus elsewhere and to groupings of competing peer institutions. A predictive analytic "dashboard" was created for practical use. RESULTS: Both models were significant at P < .001 and had similar predictive performance. In the binomial model female, underrepresented minority students, grade point average, Medical College Admission Test score, admissions committee desirability score, and most individual financial aid offers were significant (P < .05). The significant covariates were similar in the multinomial model (excluding female) and provided separate likelihoods of students enrolling at different institutional types. CONCLUSIONS: An enrollment-management-based approach would allow medical schools to better manage the number of students they admit and target recruitment efforts to improve their likelihood of success. It also performs a key institutional research function for understanding failed recruitment of highly desirable candidates.

A Hybrid Interview Model for Medical School Interviews: Combining Traditional and Multisampling Formats.

PROBLEM: Most medical schools have either retained a traditional admissions interview or fully adopted an innovative, multisampling format (e.g., the multiple mini-interview) despite there being advantages and disadvantages associated with each format. APPROACH: The University of Michigan Medical School (UMMS) sought to maximize the strengths associated with both interview formats after recognizing that combining the two approaches had the potential to capture additional, unique information about an applicant. In September 2014, the UMMS implemented a hybrid interview model with six, 6-minute short-form interviews-highly structured scenario-based encounters-and two, 30-minute semistructured long-form interviews. Five core skills were assessed across both interview formats. OUTCOMES: Overall, applicants and admissions committee members reported favorable reactions to the hybrid model, supporting continued use of the model. The generalizability coefficients for the six-station short-form and the two-interview long-form formats were estimated to be 0.470 and 0.176, respectively. Different skills were more reliably assessed by different interview formats. Scores from each format seemed to be operating independently as evidenced through moderate to low correlations (r = 0.100-0.403) for the same skills measured across different interview formats; however, after correcting for attenuation, these correlations were much higher. NEXT STEPS: This hybrid model will be revised and optimized to capture the skills most reliably assessed by each format. Future analysis will examine validity by determining whether short-form and long-form interview scores accurately measure the skills intended to be assessed. Additionally, data collected from both formats will be used to establish baselines for entering students' competencies.