Awareness of implicit bias mitigates discrimination in radiology resident selection.

OBJECTIVES: Implicit bias is common and is thought to drive discriminatory behaviour. Having previously demonstrated discrimination against specific applicant demographics by academic radiology departments in a simulated resident selection process, the authors sought to better understand the relationship between implicit bias and discrimination, as well as the potential and mechanisms for their mitigation. METHODS: A total of 51 faculty reviewers at three academic radiology departments, who had participated in a 2017 audit study in which they were shown to treat applicants differently based on race or ethnicity and physical appearance, were invited to complete testing for implicit racial and weight bias using the Implicit Association Test in 2019. Respondents were also surveyed regarding awareness of their own personal racial and weight biases, as well as any prior participation in formal diversity training. Comparisons were made between implicit bias scores and applicant ratings, as well as between diversity training and self-awareness of bias. RESULTS: A total of 31 out of 51 faculty reviewers (61%) completed and submitted results of race and weight Implicit Association Tests. A total of 74% (23/31) reported implicit anti-obese bias, concordant with discrimination demonstrated in the resident selection simulation, in which obese applicants were rated 0.40 standard deviations (SDs) lower than non-obese applicants (P < .001). A total of 71% (22/31) reported implicit anti-Black bias, discordant with application ratings, which were 0.47 SDs higher for Black than for White applicants (P < .001). A total of 84% (26/31) of participants reported feeling self-aware of potential racial bias at the time of application review, significantly higher than the 23% (7/31) reporting self-awareness of potential anti-obese bias (P < .001). Participation in formal diversity training was not associated with implicit anti-Black or anti-fat bias, nor with self-reported awareness of potential racial or weight-based bias (all P > .2). CONCLUSIONS: These findings suggest that implicit bias, as measured by the Implicit Association Test, does not inevitably lead to discrimination, and that personal awareness of implicit biases may allow their mitigation.

Diagnostic Radiologists' Participation in the American Board of Radiology Maintenance of Certification Program.

OBJECTIVE. Physicians across specialties have expressed concerns about Maintenance of Certification (MOC) programs of American Board of Medical Specialties member boards, calling for research about MOC acceptance, adoption, and value. The purpose of this study was to characterize diagnostic radiologists' participation in the American Board of Radiology (ABR) MOC program, the framework for its new Online Longitudinal Assessment program. MATERIALS AND METHODS. Practicing U.S. radiologists were identified from the Centers for Medicare & Medicaid Services Physician and Other Supplier Public Use File. Corresponding ABR diplomate certification information was obtained through the ABR public search engine. Focused on diagnostic radiologists (defined as those whose only ABR certificate is in diagnostic radiology), MOC participation rates were calculated across various physician characteristics for those whose participation was mandated by the ABR (time-limited certificates) and for those whose participation was not mandated (lifetime certificates). RESULTS. Among 20,354 included diagnostic radiologists, 11,479 (56.4%) participated in MOC. Participation rates were 99.6% (10,058/10,099) among those whose MOC was ABR mandated and 13.9% (1421/10,225) among those whose participation was not mandated (p < 0.001). The rates of nonmandated participation were higher (all p < 0.001) for academic than for non-academic radiologists (28.0% vs 11.3%), subspecialists than for generalists (17.0% vs 11.5%), and those in larger practice groups (< 10 members, 5.0%; 10-49 members, 12.6%; ≥ 50 members, 20.7%). State-level rates of nonmandated participation varied from 0.0% (South Dakota, Montana) to 32.6% (Virginia) and positively correlated with state population density (r = 0.315). CONCLUSION. Although diagnostic radiologists with time-limited certificates nearly universally participate in MOC, those with lifetime certificates (particularly general radiologists and those in smaller and nonacademic practices) participate infrequently. Low rates of nonmandated participation may reflect diplomate dissatisfaction or negative perceptions about MOC.

Women in leadership matters: Achieving diversity in radiology residency recruitment.

RATIONALE AND OBJECTIVES: To assess whether academic radiology departments and residency programs with efforts toward supporting and augmenting Diversity, Equity, and Inclusion (DEI) are associated with a higher proportion of residents from diverse backgrounds. MATERIALS AND METHODS: Program Directors within the Radiology Residency Education Research Alliance were surveyed to gather information about program characteristics, incorporation of diversity in resident recruitment, the sponsoring department's commitment to efforts at expanding diversity, and a summary of their current and past residents, staff and faculty members (academic years 2020 and 2023) with respect to a list of diversity characteristics. RESULTS: Survey response rate was 51 %. Sixty-three percent (15/24) of participating programs have departmental committees dedicated to DEI work; 46 % (11/24) of programs' departments have a Vice Chair for DEI. Sixty percent (15/24) of programs use their social media accounts to advertise their DEI programming efforts. Ninety-six percent (23/24) of programs participating in the survey use diversity factors to select candidates for their program. Women Leadership was associated with above-median diversity of residents and faculty. CONCLUSION: This study of radiology residency programs encourages a more prominent role for women in leadership positions within academic radiology departments to drive diversity and inclusion efforts.

Artificial Intelligence/Machine Learning Education in Radiology: Multi-institutional Survey of Radiology Residents in the United States.

RATIONALE AND OBJECTIVES: To evaluate radiology residents' perspectives regarding inclusion of artificial intelligence/ machine learning (AI/ML) education in the residency curriculum. MATERIALS AND METHODS: An online anonymous survey was sent to 759 residents at 21 US radiology residency programs. Resident demographics, sub-specialty interests, educational background and research experiences, as well as the awareness, availability, and usefulness of various resources for AI/ML education were collected. RESULTS: The survey response rate was 27% (209/759). A total of 74% of respondents were male, 80% were training at large university programs, and only a minority (<20) had formal education or research experience in AI/ML. All four years of training were represented (range: 20%-38%). The majority of the residents agreed or strongly agreed (83%) that AI/ML education should be a part of the radiology residency curriculum and that such education should equip them with the knowledge to troubleshoot an AI tool in practice / determine whether a tool is working as intended (82%). Among the residency programs that offer AI/ML education, the most common resources were lecture series (43%), national informatics courses (28%), and in-house/institutional courses (26%). About 24% of the residents reported no AI/ML educational offerings in their residency curriculum. Hands on AI/ML laboratory (67%) and lecture series (61%) were reported as the most beneficial or effective. The majority of the residents preferred AI/ML education offered as a continuous course spanning the radiology residency (R1 to R4) (76%), followed by mini fellowship during R4 (32%) and as a course during PGY1 (21%). CONCLUSION: Residents largely favor the inclusion of formal AI/ML education in the radiology residency curriculum, prefer hands-on learning and lectures as learning tools, and prefer a continuous AI/ML course spanning R1-R4.

For the Birds: Radiology Parallels, Finding Wellness, and Lessons Learned in Birding.

The popularity of birding has grown substantially during the COVID-19 pandemic. As an activity that can be performed year-round with social distancing in virtually any location, birding shares a striking resemblance to the practice of radiology - so much so that we feel it represents a unique wellness activity that may greatly appeal to radiologists. In this paper, we describe commonalities between birding and radiology, hopefully inspiring readers to get outdoors and commune with nature in this restorative wellness activity.

Maintenance of Certification in Radiology: Eliciting Radiologist Preferences Using a Discrete Choice Experiment.

OBJECTIVE: To quantitatively assess radiologists' preferences for Maintenance of Certification (MOC) and Continuing Certification (CC) using a survey of attitudes and perceptions. METHODS: A questionnaire that assessed attitudes and perceptions and included a discrete choice or trade-off task was developed by ACR staff in conjunction with an independent market research agency and the Survey Subcommittee of the ACR Task Force on Certification in Radiology. The trade-off exercise was integrated into this methodology to better understand the underlying utilities or preferences of the components of MOC-CC among respondents and to better enable specific recommendations on how to optimize the current program. The survey was administered via e-mail to 17,305 ACR members. The demographic and practice characteristics of the 1,994 (11.5%) respondents were similar to the ACR radiologist membership and correspond to a normal distribution. At a 95% confidence level, with a margin of error 2.1%, we believe that the respondent population fairly reflects the actual population. RESULTS: Similar proportions judged the existing program as excellent or very good (36%), or fair or poor (35%), with 27% neutral. MOC-CC was perceived more often as excellent or very good by those who were grandfathered yet still participating in MOC, were in academic practice, were in an urban setting, were older, or had a role with the ABR. In contrast, MOC-CC was more often judged as fair or poor by those who were not grandfathered, were in private practice, were in a rural setting, or were younger. The current MOC-CC program is not well regarded by diplomates, with few showing preference or acceptability. The program's reception is most sensitive to the following attributes: absence or presence of a practice quality improvement requirement, Online Longitudinal Assessment content including or excluding general radiology in addition to one's specialty and inclusion or exclusion of self-assessment as part of the CME. CONCLUSION: ACR members diverged in their attitudes toward MOC, with differences among specific demographic and practice characteristics. The current package of features of MOC-CC was widely viewed as unsatisfactory, and a more optimal feature set arose from a simulation exercise.

Can the use of deception be justified in medical education research? A point/counterpoint and case study.

Deception is a common feature of behavioral research design, although not commonly employed in the medical literature. It can promote scientific validity but is ethically controversial because it compromises subject autonomy and incurs additional costs.  In this Point/Counterpoint monograph, we review the nature of deception in research and present arguments for and against its ethical use as a research methodology in behavioral studies.  We describe the necessary guidelines, safeguards, and oversight, when deceptive methodology is considered, and report our experiences and lessons learned from conducting a multi-institutional audit study that relied upon deception of academic radiology faculty.

Should Radiology Residency Interviews Remain Virtual? Results of a Multi-institutional Survey Inform the Debate.

RATIONALE AND OBJECTIVES: The recent completion of the inaugural virtual interview season has triggered calls for the permanency of virtual interviews in the radiology resident selection process. We designed a study to assess the inaugural 2020-2021 virtual interview season and inform the anticipated debate on the future of radiology residency interviews. MATERIALS AND METHODS: Forty-four and 39-question survey instruments developed for program directors (PDs) and applicants, respectively, were distributed through the Radiology Residency Education Research Alliance to measure the demographics, experiences with technology, attitudes toward the virtual interview season and attitudes about proposed changes to the interview process. Comparisons were made between demographics and survey queries. RESULTS: PD and applicant response rates were 74% (25/34) and 45% (84/186), respectively. Eighty percent (20/25) of PDs and 76% (64/84) of applicants described the virtual interview season as excellent or very good. Sixty percent of PDs agreed or strongly agreed with the statement "The benefits of the virtual interview season outweighed the drawbacks," while 24% disagreed or strongly disagreed, and 16% were neutral. Among applicants, 80% agreed or strongly agreed, 10% disagreed or strongly disagreed, and 10% were neutral toward the same statement. Ninety-two percent of PDs noted that their rank order list performed the same or better than in years prior. Both PDs and applicants identified applicant equity and wellness as major benefits of virtual interviews, while identifying over-application and interview hoarding as significant detriments. CONCLUSION: The virtual interview provides an adequate substitute for the conventional in-person residency interview, with real and perceived benefits to applicant wellness, equity, and financial well-being. The downsides of virtual interviews, namely over-application and interview hoarding, have workable solutions.

Pediatric Program Directors Should Have an Active Presence on Twitter.

For academic pediatricians, social media has become an important avenue for professional development through continuing education, professional networking, and academic collaboration. Pediatric residency program directors have recognized additional benefits of social media engagement via program promotion and resident recruitment. The novel coronavirus disease 2019 (COVID-19) pandemic and subsequent move to virtual interviews for the 2020-2021 residency interview season have created a new urgency for pediatric program directors to establish an active social media presence, primarily as a means to engage applicants and provide them with information in lieu of cancelled away rotations and in-person interviews. Twitter is a free microblogging and social networking platform that allows real-time engagement among academic pediatricians. Here, we make the case that all pediatric program directors should have an active presence on Twitter.

USMLE Step 3 Scores Have Value in Predicting ABR Core Examination Outcome and Performance: A Multi-institutional Study.

RATIONALE AND OBJECTIVES: We analyzed multi-institutional data to determine if Step 3 performance tiers can identify radiology residents with increased risk of Core examination failure and submean performance. MATERIALS AND METHODS: We collected Step 3 scores (USMLE Step 3 or COMLEX Level 3) and American Board of Radiology (ABR) Core examination outcomes and scores for anonymized residents from 13 different Diagnostic Radiology residency programs taking the ABR Core examination between 2013 and 2019. Step 3 scores were converted to percentiles based on Z-score, with Core outcome and performance analyzed for Step 3 groups based on 50th percentile and based on quintiles. Core outcome was scored as fail when conditionally passed or failed. Core performance was measured by the percent of residents with scores below the mean. Differences between Step 3 groups for Core outcome and Core performance were statistically evaluated. RESULTS: Data were available for 342 residents. The Core examination failure rate for 121 residents with Step 3 scores <50th percentile was 19.8% (fail relative risk = 2.26), significantly higher than the 2.7% failure rate for the 221 other residents. Of 42 residents with Step 3 scores in the lowest quintile, the Core failure rate increased to 31.0% (fail relative risk = 3.52). Core performance improved with higher Step 3 quintiles. CONCLUSION: Step 3 licensing scores have value in predicting radiology resident performance on the ABR Core examination, enabling residency programs to target higher risk residents for early assessment and intervention.

An Institutional Academic and Research Track Curriculum for Radiology Residents: Development, Implementation, and Outcomes Assessment at 6 Years.

RATIONALE AND OBJECTIVES: We report the creation of a voluntary research development curriculum for radiology residents at our institution, detailing outcomes after 6 years of existence. MATERIALS AND METHODS: We developed a comprehensive Academic and Research Track (ART) curriculum for radiology residents in our department, including mentorship, monthly meetings, didactic curriculum, ongoing evaluation/feedback, protected academic time, and financial support. A literature review identified all indexed publications for our residency graduates from 6 years pre- (2007-2012) to 6 years post-ART (2013-2018) intervention. We also documented career outcomes (academic versus private practice). Summary statistics, Chi-square, and Poisson regression analyses were performed. RESULTS: ART group demonstrated significantly greater publication likelihood versus pre-ART predecessors (odds ratio [OR]: 3.59, 95% confidence intervals [CI]: 1.40-9.19; risk ratios [RR]: 1.58, 95% CI: 1.19-2.11; p= 0.01), contemporaneous non-ART cohort (OR: 3.03, 95% CI: 1.12-8.33; RR: 1.45, 95% CI: 1.08-1.96; p = 0.04), and combined pre-ART plus non-ART group (OR: 3.34, 95% CI: 1.35-8.22; RR: 1.41, 95% CI: 0.87-2.29; p = 0.012). In Poisson regression, ART was a significant factor for total, first author, and senior author publications among all graduates and among the subset which remained in academics (all p-values < 0.05). Estimators were positive in all regressions, indicating a positive effect of ART for increasing numbers of publications. CONCLUSION: ART membership was associated with significantly increased indexed publications. A greater proportion of ART members remained in academics compared to non-ART members, but did not reach statistical significance. We detail our ART structure and curriculum, which may be used as a malleable template for other radiology departments.

The Relationship Between ACR Diagnostic Radiology In-Training Examination Scores and ABR Core Examination Outcome and Performance: A Multi-Institutional Study.

PURPOSE: We analyzed multi-institutional data to understand the relationship of ACR Diagnostic Radiology In-Training Examination (DXIT) scores to ABR Core examination performance. METHODS: We collected DXIT rank scores and ABR Core examination outcomes and scores for anonymized residents from 12 different diagnostic radiology residency programs taking the ABR Core examination between 2013 and 2019. DXIT scores were grouped into quintiles based on rank score for residency year 1 (R1), residency year 2 (R2), and residency year 3 (R3) residents. Core outcome was scored as fail when conditionally passed or failed. Core performance was grouped using SD from the mean and measured by the percent of residents with scores below the mean. Differences between DXIT score quintiles for Core outcome and Core performance were statistically evaluated. RESULTS: DXIT and Core outcome data were available for 446 residents. The Core examination failure rate for the lowest quintile R1, R2, and R3 DXIT scores was 20.3%, 34.2%, and 38.0%, respectively. Core performance improved with higher R3 DXIT quintiles. Only 2 of 229 residents with R3 DXIT score ≥ 50th percentile failed the Core examination, with both failing residents having R2 DXIT scores in the lowest quintile. CONCLUSIONS: DXIT scores are useful evaluation metrics to identify a subgroup of residents at significantly higher risk for Core examination failure and another subgroup of residents at significantly lower risk for Core examination failure, with increasing predictive power with advancing residency year. These scores enable identification of approximately one-half of R3 residents whose risk of Core examination failure is negligible.

The Relationship Between US Medical Licensing Examination Step Scores and ABR Core Examination Outcome and Performance: A Multi-institutional Study.

PURPOSE: We analyzed multi-institutional data to understand the relationship of US Medical Licensing Examination (USMLE) Step scores to ABR Core examination performance to identify Step score tiers that stratify radiology residents into different Core performance groups. METHODS: We collected USMLE Step scores and ABR Core examination outcomes and scores for anonymized residents from 13 different diagnostic radiology residency programs taking the ABR Core examination between 2013 and 2019. USMLE scores were grouped into noniles using z scores and then aggregated into three tiers based on similar Core examination pass-or-fail outcomes. Core performance was grouped using standard deviation from the mean and then measured by the percent of residents with scores below the mean. Differences between Step tiers for Core outcome and Core performance were statistically evaluated (P < .05 considered significant). RESULTS: Differences in Step 1 terciles Core failure rates (45.9%, 11.9%, and 3.0%, from lowest to highest Step tiers; n = 416) and below-mean Core performance (83.8%, 54.1%, and 21.1%, respectively; n = 402) were significant. Differences in Step 2 groups Core failure rates (30.0%, 10.6%, and 2.0%, from lowest to highest Step tiers; n = 387) and below-mean Core performance (80.0%, 43.7%, and 14.0%, respectively; n = 380) were significant. Step 2 results modified Core outcome and performance predictions for residents in Step 1 terciles of varying statistical significance. CONCLUSIONS: Tiered scoring of USMLE Step results has value in predicting radiology resident performance on the ABR Core examination; effective stratification of radiology resident applicants can be done without reporting numerical Step scores.

COVID-19 Impact on Well-Being and Education in Radiology Residencies: A Survey of the Association of Program Directors in Radiology.

RATIONALE AND OBJECTIVES: The COVID-19 pandemic has forced rapid evolution of the healthcare environment. Efforts to mitigate the spread of the virus through social distancing and shelter-at-home edicts have unintended consequences upon clinical and educational missions and mental well-being of radiology departments. We sought to understand the impact of the COVID-19 pandemic on radiology residencies with respect to the educational mission and perceptions of impact on well-being. MATERIALS AND METHODS: This study was IRB exempt. An anonymous 22 question survey regarding the impact of COVID-19 pandemic on educational and clinical missions of residencies, its perceived impact upon morale of radiologists and trainees and a query of innovative solutions devised in response, was emailed to the Association of Program Directors in Radiology membership. Survey data were collected using SurveyMonkey (San Mateo, California). RESULTS: Respondents felt the COVID-19 pandemic has negatively impacted their residency programs. Regarding the educational mission impact, 70.1% (75/107) report moderate/marked negative impact and 2.8% (3/107) that educational activities have ceased. Regarding the pandemic's impact on resident morale, 44.8% (48/107) perceive moderate/marked negative effect; perceived resident morale in programs with redeployment is significantly worse with 57.1% (12/21) reporting moderate/marked decrease. Respondents overwhelmingly report adequate resident access to mental health resources during the acute phase of the pandemic (88.8%, 95/107). Regarding morale of program directors, 61% (65/106) report either mild or marked decreased morale. Program innovations reported by program directors were catalogued and shared. CONCLUSION: The COVID-19 pandemic has markedly impacted the perceived well-being and educational missions of radiology residency programs across the United States.

The USMLE Step 1 Pass/Fail Reporting Proposal: Another View.

The Association of Program Directors in Radiology recently issued a statement endorsing continued reporting of results of the United States Medical Licensing Examination (USMLE) as a three-digit score. While this position was approved by the Association of Program Directors in Radiology Board of Directors, it does not reflect the opinions of all radiology program directors. Here, we present an argument in support of reporting USMLE results as pass/fail. As a psychometric instrument, the USMLE Step 1 is designed to assess basic science knowledge and intended to inform a binary decision on licensure. Due to a steadily-increasing burden of applications to review, program directors have increasingly relied upon scores for candidate screening. Such use has multiple adverse consequences. Student focus on Step 1 systematically devalues educational content not evaluated on the exam, and the reliance on Step 1 scores almost certainly works against efforts to increase workforce diversity. Moreover, the increasing pressure of "Step 1 Mania" has negative consequences for trainee mental health and wellness. Despite the widespread use of Step 1 scores to select applicants, there are little data to correlate scores to meaningful outcomes related to patient care or clinical practice. We find the current situation untenable, and believe a necessary first step toward reform is making Step 1 a pass/fail only examination.