Addressing the Elephant in the Room: Perceptions and Treatment of Underrepresented in Medicine Physician Assistants/Associate Educators.

ABSTRACT: On Thursday, June 27, 2023, the US Supreme Court struck down race-conscious admission practices in higher education. While other demographic factors, such as the traditional Health Resources and Services Administration-disadvantaged background indicators, can be considered during the holistic admission process, explicit consideration based on race and/or ethnicity is prohibited. As a result, physician assistant/associate (PA) programs are tasked with developing novel ways to address equity, diversity, and inclusion during the admission and hiring processes. As Drumgold et al note, closing the PA workforce diversity gap is necessary to achieve health equity. Despite this, PA programs consistently struggle to attract and retain underrepresented in medicine (URiM) faculty, staff, and students. The latest PA Education Association Student Report indicates that more than 75% of applicants consider faculty and student body diversity when applying to programs. As such, addressing disparities in the recruitment, promotion, evaluation, and retention of URiM faculty is paramount. Here, the authors outline ongoing recruitment and retention challenges for URiM faculty along with institutional recommendations to ensure URiM PA faculty success and engagement.

Identifying Gender and Racial Bias in Pediatric Fellowship Letters of Recommendation: Do Word Choices Influence Interview Decisions?

OBJECTIVES: To describe linguistic differences in letters of recommendation (LORs) for pediatric fellowship candidates based on applicant and letter writer demographics and to examine if these differences influenced the decision to interview a candidate for a fellowship position. STUDY DESIGN: LORs for applicants to 8 pediatric subspecialty fellowships at a single academic center from the 2020 Match were analyzed in this cross-sectional study. Frequency of validated agentic and communal terms in each letter were determined by a language processing web application. Bias was determined as having a >5% surplus of agentic or communal terms. RESULTS: We analyzed 1521 LORs from 409 applicants: 69% were women, 28% were under-represented minorities in medicine (URM), and 50% were invited to interview. Overall, 66% of LORs were agentic biased, 16% communal biased, and 19% neutral. There was no difference in bias in LORs by an applicant's gender (woman 67% agentic vs man 62% agentic; P = .058), race, or ethnicity (non-URM 65% agentic vs URM 67% agentic; P = .660). Despite a lower frequency of agentic terms in LORs for applicants invited for interviews, when accounting for other components of an application and applicant demographics, no significant association was made between language bias in LORs and fellowship interview status. CONCLUSIONS: The frequency of agentic and communal terms in LORs for pediatric subspecialty fellowship candidates were not found to influence the decision to invite a candidate to interview. However, raising awareness of potential areas of bias within the pediatric fellowship selection process might lead to a more equitable and holistic approach to application review.

Objective Residency Applicant Assessment Using a Linear Rank Model.

OBJECTIVES: Residency applicant assessment is imperfect, with little objectivity built into the process, which, unfortunately, impacts recruitment diversity. Linear rank modeling (LRM) is an algorithm that standardizes applicant assessment to model expert judgment. Over the last 5 years, we have used LRM to assist with screening and ranking integrated plastic surgery (PRS) residency applicants. This study's primary objective was to determine if LRM scores are predictive of match success and, secondarily, to compare LRM scores between gender and self-identified race categories. DESIGN: Data was collected on applicant demographics, traditional application metrics, global intuition rank, and match success. LRM scores were calculated for screened and interviewed applicants, and scores were compared by demographic groups. Univariate logistic regression was used to evaluate the association of LRM scores and traditional application metrics with match success. SETTING: University of Wisconsin, Division of Plastic and Reconstructive Surgery. Academic institution. PARTICIPANTS: Six hundred seventeen candidates who applied to a single institution over 4 application cycles (2019-2022). RESULTS: Using area under the curve modeling, LRM score was the most predictive indicator for match success. With every one-point increase in LRM score, there was an 11% and 8.3% increase in the likelihood of screened and interviewed applicant match success (p < 0.001). An algorithm was developed to estimate the probability of match success based on LRM score. No significant differences in LRM scores were appreciated for interviewed applicant gender or self-identified race groups. CONCLUSIONS: LRM score is the most predictive indicator of match success for PRS applicants and can be used to estimate an applicant's probability of successfully matching into an integrated PRS residency. Furthermore, it provides a holistic evaluation of the applicant that can streamline the application process and improve recruitment diversity. In the future, this model could be applied to assist in the match process for other specialties.

A New Tool for Holistic Residency Application Review: Using Natural Language Processing of Applicant Experiences to Predict Interview Invitation.

PROBLEM: Reviewing residency application narrative components is time intensive and has contributed to nearly half of applications not receiving holistic review. The authors developed a natural language processing (NLP)-based tool to automate review of applicants' narrative experience entries and predict interview invitation. APPROACH: Experience entries (n = 188,500) were extracted from 6,403 residency applications across 3 application cycles (2017-2019) at 1 internal medicine program, combined at the applicant level, and paired with the interview invitation decision (n = 1,224 invitations). NLP identified important words (or word pairs) with term frequency-inverse document frequency, which were used to predict interview invitation using logistic regression with L1 regularization. Terms remaining in the model were analyzed thematically. Logistic regression models were also built using structured application data and a combination of NLP and structured data. Model performance was evaluated on never-before-seen data using area under the receiver operating characteristic and precision-recall curves (AUROC, AUPRC). OUTCOMES: The NLP model had an AUROC of 0.80 (vs chance decision of 0.50) and AUPRC of 0.49 (vs chance decision of 0.19), showing moderate predictive strength. Phrases indicating active leadership, research, or work in social justice and health disparities were associated with interview invitation. The model's detection of these key selection factors demonstrated face validity. Adding structured data to the model significantly improved prediction (AUROC 0.92, AUPRC 0.73), as expected given reliance on such metrics for interview invitation. NEXT STEPS: This model represents a first step in using NLP-based artificial intelligence tools to promote holistic residency application review. The authors are assessing the practical utility of using this model to identify applicants screened out using traditional metrics. Generalizability must be determined through model retraining and evaluation at other programs. Work is ongoing to thwart model "gaming," improve prediction, and remove unwanted biases introduced during model training.

Effect of Holistic Review, Interview Blinding, and Structured Questions in Resident Selection: Can we Predict Who Will Do Well in a Residency Interview?

OBJECTIVE: To examine the Urology residency application process, particularly the interview. Historically, the residency interview has been vulnerable to bias and not determined to be a predictor of future residency performance. Our goal is to determine the relationship between pre-interview metrics and post-interview ranking using best practices for Urology resident selection including holistic review, blinded interviews, and structured behaviorally anchored questions. METHODS: Applications were assessed on cognitive (Alpha Omega Alpha, class rank, junior year clinical clerkship grades) and non-cognitive attributes (letters of recommendation [LOR], personal statement [PS], demographics, research, personal characteristics) by reviewers blinded to USMLE scores and photograph. Interviewers were blinded to the application other than PS and LORs. Interviews consisted of a structured behaviorally anchored question (SBI) and an unstructured interview (UI). Odds ratios were determined comparing pre-interview and interview impressions. RESULTS: Fifty-one applicants were included in the analysis. USMLE step 1 score (average 245) was associated with Alpha Omega Alpha, class rank, junior year clinical clerkship, and PS. The UI score was associated with the LOR (P = .04) whereas SBI scores were not (P = .5). Faculty rank was associated with SBI, UI, and overall interview (OI) scores (P < .001). Faculty rank was also associated with LOR. Resident impression of interviewees were associated with faculty interview scores (P = .001) and faculty rank (P < .001). CONCLUSION: Traditional interviews may be biased toward application materials and may be balanced with behavioral questions. While Step 1 score does not offer additional information over other PI metrics, blinded interviews may offer discriminant validity over a PI rubric.

Recommendations to improve recruitment and retention of underrepresented in medicine trainees in neonatal-perinatal medicine.

Physicians who identify as Black, Latinx, American Indian, Pacific Islander, and certain Asian subgroups represent racial and ethnic populations that are underrepresented in medicine (URM). While the proportion of URM pediatric trainees has remained unchanged, that of Neonatal-Perinatal Medicine (NPM) fellows has decreased. Informed by the medical literature and our lived experiences, we compiled and developed a list of recommendations to support NPM fellowship programs in the recruitment, retention, and promotion of URM trainees. We describe ten recommendations that address 1) creating a culture of inclusivity and psychological safety, 2) the critical appraisal of recruitment practices and climate, and 3) an inclusive and holistic fellowship application process. The first two themes lay the foundation, while the final theme spotlights our recommendations for URM recruitment. Each recommendation is a step towards improvement in recruitment and inclusion at a program.

Exploring Characteristics of Academic General Surgery Residency Applicants: A Group Concept-Mapping Approach.

OBJECTIVE: Holistic review, which emphasizes qualitative attributes over objective measures, has been proposed as a method for selecting candidates for surgical residency in order to improve diversity in graduate medical education, and, ultimately, the field of surgery. This study seeks to articulate desirable traits of applicants as a first-step in standardizing the holistic review process. DESIGN: Using Group Concept Mapping, a web-based mixed-methods participatory research methodology, residency selection committee members were asked to 1) list desirable characteristics of applicants, 2) group these into categories, 3) rate their importance to academic/clinical success on a 5-point Likert scale (1 = not at all important, 5 = extremely important), and 4) rate the degree to which each characteristic is feasible to assess on a 3-point Likert scale (1 = not at all feasible, 3 = very feasible). Grouped characteristics submitted to hierarchical cluster analysis depicted committee's consensus about desirable qualities/criteria for applicants. Bivariate scatter-plots and pattern-matching graphics demonstrated which of these criteria were most important and reliably assessed. SETTING: A single academic general surgery residency training program in Western Pennsylvania. PARTICIPANTS: Members of the selection committee for the UPMC General Surgery Residency program who had participated in at least 1 prior cycle of applicant selection. RESULTS: Desirable characteristics of highly qualified applicants into an academic general surgery residency were clustered into domains of 1) scholarly work and research, 2) grades/formal assessments, 3) program fit, 4) behavioral assets, and 5) aspiration. Behavioral assets, which was felt to be the most important to clinical and academic success were considered to be the least feasible to reliably assess. Within this domain, initiative, being self-motivated, intellectual curiosity, work ethic, communication skills, maturity and self-awareness, and thoughtfulness were viewed as most frequently reliably assessed from the application and interview process. CONCLUSIONS: High quality applicants possess several behavioral assets that faculty deem are important to academic and clinical success. Adapting validated metrics for assessing these assets, may provide a solution for addressing subjectivity and other challenges scrutinized by critics of holistic review.

Reshaping Residency Recruitment: Achieving Alignment Between Applicants and Programs in Surgery.

OBJECTIVE: The residency recruitment process has become increasingly challenging for both applicants and program directors, in part, due to the inflation in the number of applications per student. As a result, it has become more daunting for programs to design processes that evaluate applicants holistically. Furthermore, the existing methods used to evaluate and select applicants do not necessarily predict success in residency and may inadvertently lend to gender, racial, and ethnic bias. This narrative review aims to identify innovative tools used in residency recruitment that will allow programs and applicants to better determine concordance of interests and achieve value alignment while supporting improved, objective evaluation of an applicant's unique attributes and experiences. DESIGN: PubMed was used to conduct a narrative review of recruitment strategies in admission processes of undergraduate and graduate medical education between 1975 and June 2021, using the designated Medical Subject Heading (MeSH0 terms. Inclusion criteria were established surrounding innovative tools to better objectively screen, evaluate, or select applicants. Strategies relying primarily on traditional metrics (United States Medical Licensing Examination (USMLE) scores, Alpha Omega Alpha status, and clerkship grades) were excluded. RESULTS: Forty-two articles met specific inclusion criteria. Using these articles, a framework was created with two specific aims: (1) to allow applicants and programs to express or assess interest and (2) to foster objective review of unique applicant attributes, skills, experiences, and competencies that align with program mission and values. The following five innovative tools for recruitment were identified: preference signaling, secondary applications, standardized letters of recommendation, situational judgment testing, and surgical simulation. CONCLUSIONS: As the number of applications continues to rise, strategies must be implemented to allow applicants and institutions to achieve better alignment or "fit," while also giving balanced consideration to all of an applicant's unique characteristics. A more holistic approach to applicant selection is a necessary tool in order to increase diversity and inclusion within the field of surgery.

Otolaryngology residency selection: are we doing it right?

PURPOSE OF REVIEW: This article will provide an overview of recent disruptions to the otolaryngology residency match process and conclude with questions and resources that can guide future selection system design. RECENT FINDINGS: During the implementation of the single accreditation system, the loss of osteopathic programs, reduction of osteopathic leadership positions, and lack of interest in Osteopathic Recognition represent serious threats to the profession; this has implications for the distribution of the otolaryngology workforce, plausibly decreasing healthcare access in less-populated communities. Next, the impacts of COVID-19 reverberated throughout the application process, including the reduction/elimination of away rotations, modification of application requirements, conversion to virtual interviews, and initiation of preference signaling. Soon, the transition to pass/fail scoring for the United States Medical Licensing Exam Step 1 could stimulate a paradigm shift, with a heightened emphasis on holistic review. SUMMARY: The last two match cycles have been the most dynamic and unpredictable in decades. Out of the commotion, the otolaryngology community has an opportunity for a fresh start, combining insights from past literature with recent articles compiled for this review. Moving forward, it will be advantageous to approach residency selection as a well-executed quality improvement project, requiring continuous assessment and adjustment.

Development and Validation of a Machine Learning-Based Decision Support Tool for Residency Applicant Screening and Review.

PURPOSE: Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review. METHOD: Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations. RESULTS: The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review. CONCLUSIONS: The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods.

Intersectionality of gender in recruitment and retention of the health workforce in Africa: a rapid review

Background: Despite the importance of gender and intersectionality in policy-making for human resources for health, these issues have not been given adequate consideration in health workforce recruitment and retention in Africa. Aims: The objective of this review was to show how gender intersects with other sociocultural determinants of health to create different experiences of marginalization and/or privilege in the recruitment and retention of human resources for health in Africa. Methods: This was rapid review of studies that investigated the intersectionality of gender in relation to recruitment and retention of health workers in Africa. A PubMed search was undertaken in April 2020 to identify eligible studies. Search terms used included: gender, employment, health workers, health workforce, recruitment and retention. Criteria for inclusion of studies were: primary research; related to the role of gender and intersectionality in recruitment and retention of the health workforce; conducted in Africa; quantitative or qualitative study design; and published in English. Results: Of 193 publications found, nine fulfilled the study inclusion criteria and were selected. Feminization of the nursing and midwifery profession results in difficulties in recruiting and deploying female health workers. Male domination of management positions was reported. Gender power relationship in the recruitment and retention of the health workforce is shaped by marriage and cultural norms. Occupational segregation, sexual harassment and discrimination against female health workers were reported. Conclusion: This review highlights the importance of considering gender analysis in the development of policies and programmes for human resources for health in Africa.

Assessment of Plastic Surgery Residency Applications without United States Medical Licensing Examination Step 1 Scores.

SUMMARY: The United States Medical Licensing Examination announced the changing of Step 1 score reporting from a three-digit number to pass/fail beginning on January 1, 2022. Plastic surgery residency programs have traditionally used United States Medical Licensing Examination Step 1 scores to compare plastic surgery residency applicants. Without a numerical score, the plastic surgery residency application review process will likely change. This article discusses advantages, disadvantages, and steps forward for residency programs related to the upcoming change. The authors encourage programs to continue to seek innovative methods of objectively and holistically evaluating applications.

How Do Healthcare Executives Understand and Make Decisions about Spiritual Care Provision?

OBJECTIVES: This pilot study explores how healthcare leaders understand spiritual care and how that understanding informs staffing and resource decisions. METHODS: This study is based on interviews with 11 healthcare leaders, representing 18 hospitals in 9 systems, conducted between August 2019 and February 2020. RESULTS: Leaders see the value of chaplains in terms of their work supporting staff in tragic situations and during organizational change. They aim to continue to maintain chaplaincy efforts in the midst of challenging economic realities. CONCLUSIONS: Chaplains' interactions with staff alongside patient outcomes are a contributing factor in how resources decisions are made about spiritual care.

The value of the modern vascular surgeon to the health care system: A report from the Society for Vascular Surgery Valuation Work Group.

Vascular surgeons provide an important service to the health care system. They are capable of treating a wide range of disease processes that affect both the venous and arterial systems. Their presence broadens the complexity and diversity of services that a health care system can offer both in the outpatient setting and in the inpatient setting. Because of their ability to control hemorrhage, they are critical to a safe operating room environment. The vascular surgery service line has a positive impact on hospital margin through both the direct vascular profit and loss and the indirect result of assisting other surgical and medical services in providing care. The financial benefits of a vascular service line will hold true for a wide range of alternative payment models, such as bundled payments or capitation. To fully leverage a modern vascular surgeon's skill set, significant investment is required from the health care system that is, however, associated with substantial return on the investment.

A Model for Exploring Compatibility Between Applicants and Residency Programs: Right Resident, Right Program.

Holistic review of residency applications is touted as the gold standard for selection, yet vast application numbers leave programs reliant on screening using filters such as United States Medical Licensing Examination scores that do not reliably predict resident performance and may threaten diversity. Applicants struggle to identify which programs to apply to, and devote attention to these processes throughout most of the fourth year, distracting from their clinical education. In this perspective, educators across the undergraduate and graduate medical education continuum propose new models for student-program compatibility based on design thinking sessions with stakeholders in obstetrics and gynecology education from a broad range of training environments. First, we describe a framework for applicant-program compatibility based on applicant priorities and program offerings, including clinical training, academic training, practice setting, residency culture, personal life, and professional goals. Second, a conceptual model for applicant screening based on metrics, experiences, attributes, and alignment with program priorities is presented that might facilitate holistic review. We call for design and validation of novel metrics, such as situational judgment tests for professionalism. Together, these steps could improve the transparency, efficiency and fidelity of the residency application process. The models presented can be adapted to the priorities and values of other specialties.

Implications of the United States Medical Licensing Examination Step 1 Examination Transition to Pass/Fail on Medical Students Education and Future Career Opportunities.

BACKGROUND: United States Medical Licensing Examination (USMLE) Step 1 will transition to pass/fail score by 2022. We aim to investigate US medical students' perspectives on the potential implications this transition would have on their education and career opportunities. METHODS: A cross-sectional study investigating US medical students' perspectives on the implications of transition of the USMLE Step 1 exam to pass/fail. Students were asked their preferences regarding various aspects of the USMLE Step 1 examination, including activities, educational opportunities, expenses regarding preparation for the examination, and future career opportunities. RESULTS: 215 medical students responded to the survey, 59.1% were women, 80.9% were allopathic vs. 19.1% osteopathic students. 34.0% preferred the USMLE Step 1 to be graded on a pass/fail score, whereas 53.5% preferred a numeric scale. Osteopathic vs. allopathic students were more likely to report that the pass/fail transition will negatively impact their residency match (aOR = 1.454, 95% CI: 0.515, 4.106) and specialty of choice (aOR = 3.187, 95% CI: 0.980, 10.359). 57.7% of respondents reported that the transition to a pass/fail grading system will change their study habits. CONCLUSIONS: The transition of the USMLE Step 1 to a pass/fail system has massive implications on medical students and residency programs alike. Though the majority of medical students did not prefer the USMLE Step 1 to have a pass/fail score, they must adapt their strategies to remain competitive for residency applications. Residency programs should create a composite score based off all aspects of medical students' applications in order to create a holistic and fair evaluation and ranking system.

The AAMC Standardized Video Interview: Lessons Learned From the Residency Selection Process.

Calls to change the residency selection process have increased in recent years, with many focusing on the need for holistic review and alternatives to academic metrics. One aspect of applicant performance to consider in holistic review is proficiency in behavioral competencies. The Association of American Medical Colleges (AAMC) developed the AAMC Standardized Video Interview (SVI), an online, asynchronous video interview that assesses applicants' knowledge of professionalism and their interpersonal and communication skills. The AAMC worked with the emergency medicine community to pilot the SVI. Data from 4 years of research (Electronic Residency Application Service [ERAS] 2017-2020 cycles) show the SVI is a reliable, valid assessment of these behavioral competencies. It provides information not available in the ERAS application packet, and it does not disadvantage individuals or groups. Yet despite the SVI's psychometric properties, the AAMC elected not to renew or expand the pilot in residency selection.In this Invited Commentary, the authors share lessons learned from the AAMC SVI project about introducing a new tool for use in residency selection. They recommend that future projects endeavoring to find ways to support holistic review engage all stakeholders from the start; communicate the value of the new tool early and often; make direct comparisons with existing tools; give new tools time and space to succeed; strike a balance between early adopters and broad participation; help stakeholders understand the limitations of what a tool can do; and set clear expectations about both stakeholder input and pricing. They encourage the medical education community to learn from the SVI project and to consider future partnerships with the AAMC or other specialty organizations to develop new tools and approaches that prioritize the community's needs. Finding solutions to the challenges facing residency selection should be a priority for all stakeholders.

To Work at Clinics or at Hospitals? Analysis of Family Physician Recruitment Advertising in Taiwan.

Family physicians act as gatekeepers of the healthcare system and have an indispensable role in providing holistic care in the primary care system. While previous studies had focused on the geographic maldistribution of family physicians, the current study investigated the distribution of job opportunities for family physicians by analyzing recruitment advertisements posted in medical association journals, as an indirect way to observe the marketplace demand for physicians. We collected all the recruitment advertisements for family physicians in the twelve issues of the Taiwan Medical Journal, the official organ of the Taiwan Medical Association, in 2018. In contrast to 124 new trainees annually, 739 advertisements for family physicians were posted within the entire year. After eliminating repeated advertisements, there were 302 distinct advertisements, of which hospitals accounted for 18.9% (n = 57). The job opportunities at hospitals were offered mainly by regional hospitals (n = 26) and community hospitals (n = 29), but only two by medical centers. Family physicians in Taiwan were in great demand not only by primary care clinics but also by hospitals. The role of family physicians in hospitals is worth further study.