Does Masked Interviewing Encourage Holistic Review in Residency Selection? A Mixed-Methods Study.

Problem: Medical educators increasingly champion holistic review. However, in U.S. residency selection, holistic review has been difficult to implement, hindered by a reliance on standardized academic criteria such as board scores. Masking faculty interviewers to applicants' academic files is a potential means of promoting holistic residency selection by increasing the interview's ability to make a discrete contribution to evaluation. However, little research has directly analyzed the effects of masking on how residency selection committees evaluate applicants. This mixed-methods study examined how masking interviews altered residency selection in an anesthesiology program at a large U.S. academic medical center. Intervention: During the 2019-2020 residency selection season in the University of Pennsylvania's Department of Anesthesiology & Critical Care, we masked interviewers to the major academic components of candidates' application files (board scores, transcripts, letters) on approximately half of interview days. The intent of the masking intervention was to mitigate the tendency of interviewers to form predispositions about candidates based on standardized academic criteria and thereby allow the interview to make a more independent contribution to candidate evaluation. Context: Our examination of the masking intervention used a concurrent, partially mixed, equal-status mixed-methods design guided by a pragmatist approach. We audio-recorded selection committee meetings and qualitatively analyzed them to explore how masking affected the process of candidate evaluation. We also collected independent candidate ratings from interviewers and consensus committee ratings and statistically compared ratings of candidates interviewed on masked days to ratings from conventional days. Impact: In conventional committee meetings, interviewers focused on how to reconcile academic metrics and interviews, and their evaluations of interviews were framed according to predispositions about candidates formed through perusal of application files. In masked meetings, members instead spent considerable effort evaluating candidates' "fit" and whether they came off as tactful. Masked interviewers gave halting opinions of candidates and sometimes pushed for committee leaders to reveal academic information, leading to masking breaches. Higher USMLE Step 1 score and higher medical school ranking were statistically associated with more favorable consensus rating. We found no significant differences in rating outcomes between masked and conventional interview days. Lessons learned: Elimination of academic metrics during the residency interview phase does not straightforwardly promote holistic review. While critical reflection among medical educators about the fairness and utility of such metrics has been productive, research and intervention should focus on the more proximate topic of how programs apply academic and other criteria to evaluate applicants.

Patient Involvement in Anesthesia Decision-making: A Qualitative Study of Knee Arthroplasty.

BACKGROUND: Calls to better involve patients in decisions about anesthesia-e.g., through shared decision-making-are intensifying. However, several features of anesthesia consultation make it unclear how patients should participate in decisions. Evaluating the feasibility and desirability of carrying out shared decision-making in anesthesia requires better understanding of preoperative conversations. The objective of this qualitative study was to characterize how preoperative consultations for primary knee arthroplasty arrived at decisions about primary anesthesia. METHODS: This focused ethnography was performed at a U.S. academic medical center. The authors audio-recorded consultations of 36 primary knee arthroplasty patients with eight anesthesiologists. Patients and anesthesiologists also participated in semi-structured interviews. Consultation and interview transcripts were coded in an iterative process to develop an explanation of how anesthesiologists and patients made decisions about primary anesthesia. RESULTS: The authors found variation across accounts of anesthesiologists and patients as to whether the consultation was a collaborative decision-making scenario or simply meant to inform patients. Consultations displayed a number of decision-making patterns, from the anesthesiologist not disclosing options to the anesthesiologist strictly adhering to a position of equipoise; however, most consultations fell between these poles, with the anesthesiologist presenting options, recommending one, and persuading hesitant patients to accept it. Anesthesiologists made patients feel more comfortable with their proposed approach through extensive comparisons to more familiar experiences. CONCLUSIONS: Anesthesia consultations are multifaceted encounters that serve several functions. In some cases, the involvement of patients in determining the anesthetic approach might not be the most important of these functions. Broad consideration should be given to both the applicability and feasibility of shared decision-making in anesthesia consultation. The potential benefits of interventions designed to enhance patient involvement in decision-making should be weighed against their potential to pull anesthesiologists' attention away from important humanistic aspects of communication such as decreasing patients' anxiety.

The role of the interview in residency selection: A mixed-methods study.

CONTEXT: Residency programmes invest considerable time and resources in candidate interviews as a result of their perceived ability to reveal important social traits. However, studies examining the ability of interviews to predict resident performance have shown mixed findings, and the role of the interview in candidate evaluation remains unclear. This mixed-methods study, conducted in an anaesthesiology residency programme at a large academic medical centre, examined how interviews contributed to candidate assessment and whether the addition of behavioural questions to interviews altered their role in the evaluation process. METHODS: During the 2018-2019 residency selection season in the Department of Anesthesiology and Critical Care at the University of Pennsylvania, independent ratings for each interviewee were collected from faculty interviewers. Consensus ratings subsequently established by committee were also collected. Committee meetings were audiorecorded and transcribed for qualitative analysis. Behavioural questions were integrated into half of interview days. Ratings of candidates interviewed on behavioural question days were compared statistically with those of candidates interviewed on non-behavioural question days. RESULTS: Qualitative analysis showed that interviewers heavily emphasised candidates' application files in evaluating the interviews. Interviewers focused on candidates' academic records and favoured candidates whose interview behaviours were consistent with their applications and whose applications demonstrated similarities to interviewers' traits. The addition of behavioural questions demonstrated little ability to alter these dynamics. Quantitatively, there were no significant differences in candidate rating outcomes between behavioural and non-behavioural interviewing days, whereas a higher medical school rating and higher score on the United States Medical Licensing Examination Step 1 were associated with a more favourable consensus rating. CONCLUSIONS: Residency candidates' application files predisposed interviewers' experience and evaluation of interviews, preventing the interviews from providing discrete assessments of interpersonal qualities, even when behavioural questions were included. In the continued effort to perform well-rounded assessments of residency candidates, further research and reflection on the role of interviewing in evaluation are necessary.

Improving Formative Feedback in the Operating Room Setting: Developing and Implementing an Initiative to Improve Feedback Quality and Culture.

PROBLEM: Formative feedback, given in an ongoing fashion during the learning process, is fundamental to clinical education. However, dissatisfaction with formative feedback among residents is common. Difficulties with formative feedback are intensified in the operating room (OR) setting due to fast pace, space limitations, and frequent rotation of residents and attendings. APPROACH: In the anesthesiology and critical care department at the University of Pennsylvania Perelman School of Medicine, the authors launched the Feedback Moment initiative from January 2018 to May 2018 in which 24 first-year residents and attendings were given a short series of prompts designed to facilitate regular, high-quality formative feedback. The authors conducted semistructured interviews with residents before and after the initiative to evaluate its impact. OUTCOMES: In baseline interviews, 18 participating residents stressed the importance of formative feedback but described feeling unsure of their performance due to lack of ongoing constructive input from attendings. They felt hesitant to approach attendings for feedback due to a desire not to interrupt OR workflow or appear incompetent. In follow-up interviews, residents described the initiative as helping to normalize constructive formative feedback but difficult to execute regularly due to OR workflow issues and frequent rotation of attendings with varying approaches. NEXT STEPS: Challenges faced by participants in this initiative highlight several considerations for effective OR-based formative feedback. Alternative timings for initiating feedback must be considered in light of the hectic nature of the OR workflow. Residents should be equipped with the skills necessary to adapt to varying practice patterns and frequent rotation between attendings, while attendings should be trained to provide a clear rationale for constructive feedback that allows residents to quickly adapt to practice variation. Finally, establishing clear goals among resident-attending pairs is critical to ensuring that formative feedback given in necessarily brief sessions is focused and productive.