Innovation in Residency Selection: The AAMC Standardized Video Interview.

PURPOSE: Innovative tools are needed to shift residency selection toward a more holistic process that balances academic achievement with other competencies important for success in residency. The authors evaluated the feasibility of the AAMC Standardized Video Interview (SVI) and evidence of the validity of SVI total scores. METHOD: The SVI, developed by the Association of American Medical Colleges, consists of six questions designed to assess applicants' interpersonal and communication skills and knowledge of professionalism. Study 1 was conducted in 2016 for research purposes. Study 2 was an operational pilot administration in 2017; SVI data were available for use in residency selection by emergency medicine programs for the 2018 application cycle. Descriptive statistics, correlations, and standardized mean differences were used to examine data. RESULTS: Study 1 included 855 applicants; Study 2 included 3,532 applicants. SVI total scores were relatively normally distributed. There were small correlations between SVI total scores and United States Medical Licensing Examination Step exam scores, Alpha Omega Alpha Honor Medical Society membership, and Gold Humanism Honor Society membership. There were no-to-small group differences in SVI total scores by gender and race/ethnicity, and small-to-medium differences by applicant type. CONCLUSIONS: Findings provide initial evidence of the validity of SVI total scores and suggest that these scores provide different information than academic metrics. Use of the SVI, as part of a holistic screening process, may help program directors widen the pool of applicants invited to in-person interviews and may signal that programs value interpersonal and communication skills and professionalism.

Association of MCAT scores obtained with standard vs extra administration time with medical school admission, medical student performance, and time to graduation.

IMPORTANCE: Individuals with documented disabilities may receive accommodations on the Medical College Admission Test (MCAT). Whether such accommodations are associated with MCAT scores, medical school admission, and medical school performance is unclear. OBJECTIVE: To determine the comparability of MCAT scores obtained with standard vs extra administration time with respect to likelihood of acceptance to medical school and future medical student performance. DESIGN, SETTING, AND PARTICIPANTS: Retrospective cohort study of applicants to US medical schools for the 2011-2013 entering classes who reported MCAT scores obtained with standard time (n = 133,962) vs extra time (n = 435), and of students who matriculated in US medical schools from 2000-2004 who reported MCAT scores obtained with standard time (n = 76,262) vs extra time (n = 449). EXPOSURES: Standard or extra administration time during MCAT. MAIN OUTCOMES AND MEASURES: Primary outcome measures were acceptance rates at US medical schools and graduation rates within 4 or 5 years after matriculation. Secondary outcome measures were pass rates on the United States Medical Licensing Examination (USMLE) Step examinations and graduation rates within 6 to 8 years after matriculation. RESULTS: Acceptance rates were not significantly different for applicants who had MCAT scores obtained with standard vs extra time (44.5% [59,585/133,962] vs 43.9% [191/435]; difference, 0.6% [95% CI, -4.1 to 5.3]). Students who tested with extra time passed the Step examinations on first attempt at significantly lower rates (Step 1, 82.1% [344/419] vs 94.0% [70,188/74,668]; difference, 11.9% [95% CI, 9.6% to 14.2%]; Step 2 CK, 85.5% [349/408] vs 95.4% [70,476/73,866]; difference, 9.9% [95% CI, 7.8% to 11.9%]; Step 2 CS, 92.0% [288/313] vs 97.0% [60,039/61,882]; difference, 5.0% [95% CI, 3.1% to 6.9%]). They also graduated from medical school at significantly lower rates at different times (4 years, 67.2% [285/424] vs 86.1% [60,547/70,305]; difference, 18.9% [95% CI, 15.6% to 22.2%]; 5 years, 81.6% [346/424] vs 94.4% [66,369/70,305]; difference, 12.8% [95% CI, 10.6% to 15.0%]; 6 years, 85.4% [362/424] vs 95.8% [67,351/70,305]; difference, 10.4% [95% CI, 8.5% to 12.4%]; 7 years, 88.0% [373/424] vs 96.2% [67,639/70,305]; difference, 8.2% [95% CI, 6.4% to 10.1%]; 8 years, 88.4% [375/424] vs 96.5% [67,847/70,305]; difference, 8.1% [95% CI, 6.3% to 9.8%]). These differences remained after controlling for MCAT scores and undergraduate grade point averages. CONCLUSIONS AND RELEVANCE: Among applicants to US medical schools, those with MCAT scores obtained with extra test administration time, compared with standard administration time, had no significant difference in rate of medical school admission but had lower rates of passing the USMLE Step examinations and of medical school graduation within 4 to 8 years after matriculation. These findings raise questions about the types of learning environments and support systems needed by students who test with extra time on the MCAT to enable them to succeed in medical school.

The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants.

PURPOSE: Most research examining the predictive validity of the Medical College Admission Test (MCAT) has focused on the relationship between MCAT scores and scores on the United States Medical Licensing Examination Step exams. This study examined whether MCAT scores predict students' unimpeded progress toward graduation (UP), which the authors defined as not withdrawing or being dismissed for academic reasons, graduating within five years of matriculation, and passing the Step 1, Step 2 Clinical Knowledge, and Step 2 Clinical Skills exams on the first attempt. METHOD: Students who matriculated during 2001-2004 at 119 U.S. medical schools were included in the analyses. Logistic regression analyses were used to estimate the relationships between UP and MCAT total scores alone, undergraduate grade point averages (UGPAs) alone, and UGPAs and MCAT total scores together. All analyses were conducted at the school level and were considered together to evaluate relationships across schools. RESULTS: The majority of matriculants experienced UP. Together, UGPAs and MCAT total scores predicted UP well. MCAT total scores alone were a better predictor than UGPAs alone. Relationships were similar across schools; however, there was more variability across schools in the relationship between UP and UGPAs than between UP and MCAT total scores. CONCLUSIONS: The combination of UGPAs and MCAT total scores performs well as a predictor of UP. Both UGPAs and MCAT total scores are strong predictors of academic performance in medical school through graduation, not just the first two years. Further, these relationships generalize across medical schools.

An overview of the medical school admission process and use of applicant data in decision making: what has changed since the 1980s?

PURPOSE: To investigate current medical school admission processes and whether they differ from those in 1986 when they were last reviewed by the Association of American Medical Colleges (AAMC). METHOD: In spring 2008, admission deans from all MD-granting U.S. and Canadian medical schools using the Medical College Admission Test (MCAT) were invited to complete an online survey that asked participants to describe their institution's admission process and to report the use and rate the importance of applicant data in making decisions at each stage. RESULTS: The 120 responding admission officers reported using a variety of data to make decisions. Most indicated using interviews to assess applicants' personal characteristics. Compared with 1986, there was an increase in the emphasis placed on academic data during pre-interview screening. While GPA data were among the most important data in decision making at all stages in 1986, data use and importance varied by the stage of the process in 2008: MCAT scores and undergraduate GPAs were rated as the most important data for deciding whom to invite to submit secondary applications and interview, whereas interview recommendations and letters of recommendation were rated as the most important data in deciding whom to accept. CONCLUSIONS: This study underscores the complexity of the medical school admission process and suggests increased use of a holistic approach that considers the whole applicant when making admission decisions. Findings will inform AAMC initiatives focused on transforming admission processes.

Information search and decision making: effects of age and complexity on strategy use.

The impact of task complexity on information search strategy and decision quality was examined in a sample of 135 young, middle-aged, and older adults. We were particularly interested in the competing roles of fluid cognitive ability and domain knowledge and experience, with the former being a negative influence and the latter being a positive influence on older adults' performance. Participants utilized 2 decision matrices, which varied in complexity, regarding a consumer purchase. Using process tracing software and an algorithm developed to assess decision strategy, we recorded search behavior, strategy selection, and final decision. Contrary to expectations, older adults were not more likely than the younger age groups to engage in information-minimizing search behaviors in response to increases in task complexity. Similarly, adults of all ages used comparable decision strategies and adapted their strategies to the demands of the task. We also examined decision outcomes in relation to participants' preferences. Overall, it seems that older adults utilize simpler sets of information primarily reflecting the most valued attributes in making their choice. The results of this study suggest that older adults are adaptive in their approach to decision making and that this ability may benefit from accrued knowledge and experience.

Ease of counterfactual thought generation moderates the relationship between need for cognition and punitive responses to crime.

Punitive responses to crime have been linked to a relatively low need for cognition (NFC). Sargent's (2004) findings suggest that this relationship is due to a relatively complex attributional system, employed by high-NFC individuals, which permits them to recognize potential external or situational causes of crime. However, high-NFC individuals may also be more likely to engage in counterfactual thinking, which has been linked to greater judgments of blame and responsibility. Three studies examine the relationship between trait and state NFC and punitiveness in light of counterfactual thinking. Results suggest that the ease of generating upward counterfactuals in response to an unfortunate crime moderates the NFC-punitiveness relationship, such that high-NFC individuals are less punitive than low-NFC individuals only when counterfactual thoughts are relatively difficult to generate. These findings are discussed in light of punishment theory and their possible implications with regard to the legal system.