Analysis of the medical residency matching algorithm to validate and improve equity.

Algorithms are becoming prevalent but are often opaque and need external validation to assess whether or not they meet their purported objectives. The purpose of this study is to validate, using the limited information available, the algorithm used by the National Resident Matching Program (NRMP) whose intention is to match applicants to medical residencies based on applicants' prioritized preferences. The methodology involved first using randomized computer-generated data to overcome the inaccessible proprietary data on applicant and program rankings. Simulations using these data were run through the compiled algorithm's procedures to obtain match outcomes. The study's findings are that the current algorithm's matches are related to program input but not to applicant input, the applicant's prioritized ranking of programs. A modified algorithm with student input as the primary factor is then developed and run using the same data, resulting in match outcomes that are related to both applicant and program inputs, improving equity.

Comparison of Program-centric vs Student-centric National Resident Matching Algorithms.

Importance: The current program-centric algorithm for the National Resident Matching Program (NRMP) primarily uses the program's ranking of students to determine a match. Concerns that the existing algorithm favors programs over students, recent findings that the program's ranking of applicants is not associated with resident performance, and disruptions of existing screening methods and metrics have prompted reevaluation of the current algorithm relative to a student-centric algorithm, in which student ranking of programs is primary and program ranking of students is secondary. Objective: To compare program-centric and student-centric algorithms for the NRMP participants. Design, Setting, and Participants: This cross-sectional study used randomized computer-generated data reflecting the NRMP match for 2018, 2019, and 2020, capturing more than 50 000 students and more than 4000 programs in 23 specialties, to compare the 2 algorithms. Exposures: The same simulated students, programs, and rankings were exposed to the 2 algorithms, running 2300 simulations in the overall analysis and 1000 simulations in each of 23 specialties. Main Outcomes and Measures: The percentage of students who did and did not match, the percentage of students who matched to their top-ranked and top-5-ranked programs, and the program's rank of the last student matched per position were examined. Results: The 2 algorithms were not different in percentage of students matched overall (eg, for 2020, program-centric: 59% [95% CI, 57%-61%]; student-centric: 58% [95% CI, 56%-60%]; P = .73). The student-centric algorithm, relative to the program-centric algorithm, matched a significantly higher percentage of students to their first-ranked program (eg, for 2020, 50% [95% CI, 48%-52%] vs 14% [95% CI, 13%-15%]; P < .001) and to their top-5-ranked programs (eg, for 2020, 60% [95% CI, 58%-62%] vs 46% [95% CI, 44%-48%]; P < .001). However, the last position was filled with students who had lower program rankings in the student-centric algorithm vs the program-centric algorithm (2 [95% CI, 1-2] vs 8 [95% CI, 6-10]; P < .001). Conclusions and Relevance: In this study, the 2 algorithms were not different in the percentage of students matched overall. However, the student-centric algorithm matched a significantly higher percentage of students to their preferred programs. The program-centric algorithm was associated with a lower program's last matched student rank. Further research is needed on the algorithms' associations with cost and time demands in the match, postmatch resident and program performance, and fit with a changing environment.