Evaluating the Impact of Coaching Through the Transition to Residency.

BACKGROUND: Coaching has been proposed to support the transition to residency. Clarifying its impact will help define its value and best use. OBJECTIVE: To explore the experiences of residents working with coaches through the residency transition. DESIGN: A cohort comparison survey compared experiences of a coached resident cohort with coaches to the prior, uncoached cohort. PARTICIPANTS: Post-graduate year (PGY)-2 residents in internal medicine, obstetrics and gynecology, emergency medicine, and pathology at a single academic center. INTERVENTIONS: Faculty trained as coaches had semi-structured meetings with graduating medical students and residents throughout the PGY-1 year. MAIN MEASURES: An online anonymous survey assessed effects of coaching on measures of self-directed learning, professional development, program support and impact of coaching using existing scales (2-item Maslach Burnout Inventory, Brief Resilient Coping Scale, 2-item Connor-Davidson Resilience Scale, Stanford Professional Fulfillment Inventory), and novel measures adapted for this survey. Bivariate analyses (t-tests and chi-square tests) compared cohort responses. MANOVA assessed the effects of coaching, burnout and their interactions on the survey domains. KEY RESULTS: Of 156 PGY2 residents, 86 (55%) completed the survey. More residents in the "un-coached" cohort reported burnout (69%) than the "coached" cohort (51%). Burnout was significantly and negatively associated (F = 3.97 (df 7, 75); p < .001) with the learning and professional development outcomes, while being coached was significantly and positively associated with those outcomes (F = 5.54 (df 9, 75); p < .001). Significant interaction effects were found for goal-setting attitudes, professional fulfillment, and perceived program career support such that the positive differences in these outcomes between coached and un-coached residents were greater among burned out residents. Coached residents reported a positive impact of coaching across many domains. CONCLUSIONS: Residents experiencing coaching reported better professional fulfillment and development outcomes, with more pronounced differences in trainees experiencing burnout. Coaching is a promising tool to support a fraught professional transition.

Understanding medical student paths to communication skills expertise using latent profile analysis.

PURPOSE: To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students. MATERIALS AND METHODS: Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items. RESULTS: Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles. CONCLUSION: Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.

"I Don't Trust It": Use of a Routine OSCE to Identify Core Communication Skills Required for Counseling a Vaccine-Hesitant Patient.

BACKGROUND: Vaccine hesitancy is challenging for clinicians and of increasing concern since COVID-19 vaccination rollout began. Standardized patients (SPs) provide an ideal method for assessing resident physicians' current skills, providing opportunity to practice and gain immediate feedback, while also informing evaluation of curriculum and training. As such, we designed and implemented an OSCE station where residents were tasked with engaging and educating a vaccine-hesitant patient. AIM: Describe residents' vaccine counseling practices, core communication and interpersonal skills, and effectiveness in meeting the objectives of the case. Explore how effectiveness in overcoming vaccine hesitancy may be associated with communication and interpersonal skills in order to inform educational efforts. SETTING: Annual OSCE at a simulation center. PARTICIPANTS: 106 internal medicine residents (51% PGY1, 49% PGY2). PROGRAM DESCRIPTION: Residents participated in an annual residency-wide, multi-station OSCE, one of which included a Black, middle-aged, vaccine-hesitant male presenting for a routine video visit. Residents had 10 min to complete the encounter, during which they sought to educate, explore concerns, and make a recommendation. After each encounter, faculty gave residents feedback on their counseling skills and reviewed best practices for effective communication on the topic. SPs completed a behaviorally anchored checklist (30 items across 7 clinical skill domains and 2 measures of trust in the vaccine's safety and resident) which will inform future curriculum. PROGRAM EVALUATION: Fifty-five percent (SD: 43%) of the residents performed well on the vaccine-specific education domain. PGY2 residents scored significantly higher on two of the seven domains compared to PGY1s (patient education/counseling-PGY1: 35% (SD: 36%) vs. PGY2: 52% (SD: 41%), p = 0.044 and activation-PGY1: 37% (SD: 45%) vs. PGY2: 59% (SD: 46%), p = 0.016). In regression analyses, education/counseling and vaccine-specific communication skills were strongly, positively associated with trust in the resident and in the vaccine's safety. A review of qualitative data from the SPs' perspective suggested that low performers did not use patient-centered communication skills. DISCUSSION: This needs assessment suggests that many residents needed in-the-moment feedback, additional education, and vaccine-specific communication practice. Our program plans to reinforce evidence-based practices physicians can implement for vaccine hesitancy through ongoing curriculum, practice, and feedback. This type of needs assessment is replicable at other institutions and can be used, as we have, to ultimately shed light on next steps for programmatic improvement.

The Telemedicine Takeover: Lessons Learned During an Emerging Pandemic.

Introduction: Health care systems rose to the challenges of COVID-19 by creating or expanding telehealth programs to ensure that patients could access care without an in-person appointment. Traditionally, physicians receive limited formal telemedicine training, making preparedness for this transition uneven. To describe challenges to and attitudes toward providing virtual patient care, we distributed a survey to physicians within our system who largely had no formal prior training/experience with telemedicine, but transitioned to routine telemedicine use. Data collected are then used to offer actionable recommendations for health system leaders and medical educators. Materials and Methods: Surveys were distributed to all faculty outpatient general internal medicine physicians working at any New York University Langone Health, New York City Health + Hospitals/Bellevue and Gouverneur, and the VA NY Harbor Health System (n = 378) in mid-2020. Survey items consisted of Likert and open-ended questions related to experience with televisits (13 items) and attitudes toward care (24 items). Results: Telehealth-related challenges varied by site and modality. Primary challenges included establishing a connection from the patient's (98%) or physician's end (84%) and difficulty in the following domains: working with team members (39%), physical examinations (95%), establishing new patient relationships (70%), and taking history (40%), among others. In thematic analysis, significant themes with illustrative qualitative commentary emerged related to technological challenges, new systems issues, and new patient/physician communication experiences. Discussion: Experience differences were rooted in the type of technology employed. Safety-net practices conducted mostly telephonic visits, whereas private outpatient sites utilized video, despite both using identical electronic medical records. As we consider a "new normal" and prolonged community transmission of COVID-19 infection, it is essential to establish telemedicine training, tools, and protocols that meet the needs of both patients and physicians.

Development and maintenance of a medical education research registry.

BACKGROUND: Medical Education research suffers from several methodological limitations including too many single institution, small sample-sized studies, limited access to quality data, and insufficient institutional support. Increasing calls for medical education outcome data and quality improvement research have highlighted a critical need for uniformly clean and easily accessible data. Research registries may fill this gap. In 2006, the Research on Medical Education Outcomes (ROMEO) unit of the Program for Medical Innovations and Research (PrMEIR) at New York University's (NYU) Robert I. Grossman School of Medicine established the Database for Research on Academic Medicine (DREAM). DREAM is a database of routinely collected, de-identified undergraduate (UME, medical school leading up to the Medical Doctor degree) and graduate medical education (GME, residency also known as post graduate education leading to eligibility for specialty board certification) outcomes data available, through application, to researchers. Learners are added to our database through annual consent sessions conducted at the start of educational training. Based on experience, we describe our methods in creating and maintaining DREAM to serve as a guide for institutions looking to build a new or scale up their medical education registry. RESULTS: At present, our UME and GME registries have consent rates of 90% (n = 1438/1598) and 76% (n = 1988/2627), respectively, with a combined rate of 81% (n = 3426/4225). 7% (n = 250/3426) of these learners completed both medical school and residency at our institution. DREAM has yielded a total of 61 individual studies conducted by medical education researchers and a total of 45 academic journal publications. CONCLUSION: We have built a community of practice through the building of DREAM and hope, by persisting in this work the full potential of this tool and the community will be realized. While researchers with access to the registry have focused primarily on curricular/ program evaluation, learner competency assessment, and measure validation, we hope to expand the output of the registry to include patient outcomes by linking learner educational and clinical performance across the UME-GME continuum and into independent practice. Future publications will reflect our efforts in reaching this goal and will highlight the long-term impact of our collaborative work.

"I Cannot Take This Any More!": Preparing Interns to Identify and Help a Struggling Colleague.

BACKGROUND: Few programs train residents in recognizing and responding to distressed colleagues at risk for suicide. AIM: To assess interns' ability to identify a struggling colleague, describe resources, and recognize that physicians can and should help colleagues in trouble. SETTING: Residency programs at an academic medical center. PARTICIPANTS: One hundred forty-five interns. PROGRAM DESIGN: An OSCE case was designed to give interns practice and feedback on their skills in recognizing a colleague in distress and recommending the appropriate course of action. Embedded in a patient "sign-out" case, standardized health professionals (SHP) portrayed a resident with depressed mood and an underlying drinking problem. The SHP assessed intern skills in assessing symptoms and directing the resident to seek help. PROGRAM EVALUATION: Interns appreciated the opportunity to practice addressing this situation. Debriefing the case led to productive conversations between faculty and residents on available resources. Interns' skills require further development: while 60% of interns asked about their colleague's emotional state, only one-third screened for depression and just under half explored suicidal ideation. Only 32% directed the colleague to specific resources for his depression (higher among those that checked his emotional state, 54%, or screened for depression, 80%). DISCUSSION: This OSCE case identified varying intern skill levels for identifying and assessing a struggling colleague while also providing experiential learning and supporting a culture of addressing peer wellness.

A Qualitative Evaluation of Mental Health Clinic Staff Perceptions of Barriers and Facilitators to Treating Tobacco Use.

Introduction: Veterans with mental health disorders smoke at high rates, but encounter low rates of tobacco treatment. We sought to understand barriers and facilitators to treating tobacco use in VA mental health clinics. Methods: This qualitative study was part of a trial evaluating a telephone care coordination program for smokers using mental health services at six VA facilities. We conducted semi-structured interviews with 14 staff: 12 mental health clinic staff working at the parent study's intervention sites (n = 6 psychiatrists, three psychologists, two social workers, one NP), as well as one psychiatrist and one psychologist on the VA's national tobacco advisory committee. Interviews were transcribed and inductively coded to identify themes. Results: Five "barriers" themes emerged: (1) competing priorities, (2) patient challenges/resistance, (3) complex staffing/challenging cross-discipline coordination, (4) mixed perceptions about whether tobacco is a mental health care responsibility, and (5) limited staff training/comfort in treating tobacco. Five "facilitators" themes emerged: (1) reminding mental health staff about tobacco, (2) staff belief in the importance of addressing tobacco, (3) designating a cessation medication prescriber, (4) linking tobacco to mental health outcomes and norms, and (5) limiting mental health staff burden. Conclusions: VA mental health staff struggle with knowing that tobacco use is important, but they face competing priorities, encounter patient resistance, are conflicted on their role in addressing tobacco, and lack tobacco training. They suggested strategies at multiple levels that would help overcome those barriers that can be used to design interventions that improve tobacco treatment delivery for mental health patients. Implications: This study builds upon the existing literature on the high rates of smoking, but low rates of treatment, in people with mental health diagnoses. This study is one of the few qualitative evaluations of mental health clinic staff perceptions of barriers and facilitators to treating tobacco. The study results provide a multi-level framework for developing strategies to improve the implementation of tobacco treatment programs in mental health clinics.

Assessment of Abilities of Gastroenterology Fellows to Provide Information to Patients With Liver Disease.

BACKGROUND & AIMS: Patient education is critical in ensuring patient compliance and good health outcomes. Fellows must be able to effectively communicate with their patients, delivering enough information for the patient to understand their medical problem and maximize patient compliance. We created an objective structured clinical examination (OSCE) with 4 liver disease cases to assess fellows' knowledge and ability to inform standardized patients (SPs) about their clinical condition. METHODS: We developed 4 cases highlighting different aspects of liver disease and created a 4-station OSCE: hepatitis B, acute hepatitis C, new diagnosis of cirrhosis, and an end-stage cirrhotic nontransplant candidate. The SP with hepatitis B was minimizing the fact that she could not read English. The acute hepatitis C SP was a nursing student who is afraid that having hepatitis C might jeopardize her career. The SP with the new diagnosis of alcoholic cirrhosis needed to stop drinking, and the end-stage liver disease patient had to grapple with his advanced directives. Twelve fellows from 4 GI training programs participated. Our focus was to assess the fellows' knowledge about liver diseases and the Accreditation Council for Graduate Medical Education competencies of health literacy, shared decision making, advanced directives, and goals of care. The goal for the fellows was to communicate effectively with the SPs, and acknowledge that each patient had an emotionally charged issue to overcome. The SPs used a checklist to rate fellows' performance. Faculty and the SPs observed the cases and provided feedback. The fellows were surveyed on their performance regarding the case. RESULTS: The majority of fellows were able to successfully summarize findings and discuss a plan with the patient in the new diagnosis of cirrhosis (76.92%) and hepatitis C case (100%), but were less successful in the hepatitis B case (30.77%) and the end-of-life case (41.67%). Overall, a small percentage of fellows reflected that they did a good job (22%-33%), except at the end-of-life case (67%). The fellows' greatest challenge was trying to cover a lot of information in a single outpatient visit. CONCLUSIONS: Caring for patients with liver diseases can be complex and time consuming. The patients and fellows' observations were discordant in several areas: for example, the fellows believed they excelled in the end-of-life case, but the SP thought only a small percentage of fellows were able to successfully summarize and discuss the plan. This discrepancy and others highlight important areas of focus in training programs. OSCEs are important to help the fellows facilitate striking the right balance of information delivery and empathy, and this will lead to better patient education, compliance, rapport, and satisfaction.

Opioid vs nonopioid prescribers: Variations in care for a standardized acute back pain case.

BACKGROUND: Opioid analgesics are effective and appropriate therapy for many types of acute pain. Epidemiologic evidence supports a direct relationship between increased opioid prescribing and increases in opioid use disorders and overdoses. OBJECTIVE: To tailor our residency curriculum, we designed and fielded an unannounced standardized patient (USP) case involving a patient with acute back pain who is requesting Vicodin (5/325 mg). We describe residents' case management and examine whether their management decisions, including opioid prescribing, were related to their core clinical skills. METHODS: Results are based on 50 (USP) visits with residents in 2 urban primary care clinics. Highly trained USPs portrayed a patient with acute lower back pain who was taking leftover Vicodin with effective pain relief but was running out. We describe how residents managed this case, using both USP report and chart review data, and compare summary clinical skills scores between those who prescribed Vicodin and those who did not. RESULTS: Of the 50 residents, 18 prescribed Vicodin (10-60 pills). Among those who did not prescribe (32/50), most (50%) prescribed ibuprofen. Eighty-three percent of the prescribers and 72% of nonprescribers ordered physical therapy (nonsignificant). Of the 18 prescribers, 13 documented checking the prescription monitoring database. Prescribers had significantly better communication scores than nonprescribers (relationship development: 80% vs. 58% well done, P = .029; patient education: 59% vs. 31% well done, P = .018). Assessment summary scores were also higher (60% vs. 46%) but not significantly (P = .060). Patient satisfaction and activation scores were higher in the prescribers than nonprescribers (71% vs. 39%, P = .004 and 48% vs. 26%, P = .034, respectively). CONCLUSIONS: Most Vicodin prescribers did not follow prescribing guidelines, and they demonstrated better communication and assessment skills than the nonprescribers. Results suggest the need to guide residents in using a systematic approach to prescribing opioids safely and to develop an acceptable alternative pain management plan when they decide against prescribing.

Professionalism Training For Surgical Residents: Documenting the Advantages of a Professionalism Curriculum.

OBJECTIVES: Professionalism education is a vital component of surgical training. This research attempts to determine whether an annual, year-long professionalism curriculum in a large surgical residency can effectively change professionalism attitudes. SUMMARY OF BACKGROUND DATA: The ACGME mandated 6 competencies in 2003. The competencies of Professionalism and Interpersonal/Professional Communication Skills had never been formally addressed in surgical resident education in the past. METHODS: A professionalism curriculum was developed focusing on specific resident professionalism challenges: admitting mistakes, effective communication with colleagues at all levels, delivering the news of an unexpected death, interdisciplinary challenges of working as a team, the cultural challenge of obtaining informed consent through an interpreter, and the stress of surgical practice on you and your family. These professionalism skills were then evaluated with a 6-station Objective Structured Clinical Examination (OSCE). Identical OSCE scenarios were administered to 2 cohorts of surgical residents: in 2007 (before instituting the professionalism curriculum in 2008) and again in 2014. Surgical residents were rated by trained Standardized Patients according to a behaviorally anchored professionalism criteria checklist. RESULTS: An analysis of variance was conducted of overall OSCE professionalism scores (% well done) as the dependent variable for the 2 resident cohorts (2007 vs 2014). The 2007 residents received a mean score of 38% of professionalism items "well done" (SD 9%) and the 2014 residents received a mean 59% "well done" (SD 8%). This difference is significant (F = 49.01, P < .001). CONCLUSIONS: Professionalism education has improved surgical resident understanding, awareness, and practice of professionalism in a statistically significant manner from 2007 to 2014. This documented improvement in OSCE performance reflects the value of a professionalism curriculum in the care of the patients we seek to serve.

Charting a Key Competency Domain: Understanding Resident Physician Interprofessional Collaboration (IPC) Skills.

BACKGROUND: Interprofessional collaboration (IPC) is essential for quality care. Understanding residents' level of competence is a critical first step to designing targeted curricula and workplace learning activities. In this needs assessment, we measured residents' IPC competence using specifically designed Objective Structured Clinical Exam (OSCE) cases and surveyed residents regarding training needs. METHODS: We developed three cases to capture IPC competence in the context of physician-nurse collaboration. A trained actor played the role of the nurse (Standardized Nurse - SN). The Interprofessional Education Collaborative (IPEC) framework was used to create a ten-item behaviorally anchored IPC performance checklist (scored on a three-point scale: done, partially done, well done) measuring four generic domains: values/ethics; roles/responsibilities; interprofessional communication; and teamwork. Specific skills required for each scenario were also assessed, including teamwork communication (SBAR and CUS) and patient-care-focused tasks. In addition to evaluating IPC skills, the SN assessed communication, history-taking and physical exam skills. IPC scores were computed as percent of items rated well done in each domain (Cronbach's alpha > 0.77). Analyses include item frequencies, comparison of mean domain scores, correlation between IPC and other skills, and content analysis of SN comments and resident training needs. RESULTS: One hundred and seventy-eight residents (of 199 total) completed an IPC case and results are reported for the 162 who participated in our medical education research registry. IPC domain scores were: Roles/responsibilities mean = 37 % well done (SD 37 %); Values/ethics mean = 49 % (SD 40 %); Interprofessional communication mean = 27 % (SD 36 %); Teamwork mean = 47 % (SD 29 %). IPC was not significantly correlated with other core clinical skills. SNs' comments focused on respect and IPC as a distinct skill set. Residents described needs for greater clarification of roles and more workplace-based opportunities structured to support interprofessional education/learning. CONCLUSIONS: The IPC cases and competence checklist are a practical method for conducting needs assessments and evaluating IPC training/curriculum that provides rich and actionable data at both the individual and program levels.

Panel Management to Improve Smoking and Hypertension Outcomes by VA Primary Care Teams: A Cluster-Randomized Controlled Trial.

BACKGROUND: Panel Management can expand prevention and chronic illness management beyond the office visit, but there is limited evidence for its effectiveness or guidance on how best to incorporate it into practice. OBJECTIVE: We aimed to test the effectiveness of incorporating panel management into clinical practice by incorporating Panel Management Assistants (PMAs) into primary care teams with and without panel management education. DESIGN: We conducted an 8-month cluster-randomized controlled trial of panel management for improving hypertension and smoking cessation outcomes among veterans. PATRICIPANTS: Twenty primary care teams from the Veterans Affairs New York Harbor were randomized to control, panel management support, or panel management support plus education groups. Teams included 69 clinical staff serving 8,153 hypertensive and/or smoking veterans. INTERVENTIONS: Teams assigned to the intervention groups worked with non-clinical Panel Management Assistants (PMAs) who monitored care gaps and conducted proactive patient outreach, including referrals, mail reminders and motivational interviewing by telephone. MAIN MEASURES: Measurements included mean systolic and diastolic blood pressure, proportion of patients with controlled blood pressure, self-reported quit attempts, nicotine replacement therapy (NRT) prescriptions, and referrals to disease management services. KEY RESULTS: Change in mean blood pressure, blood pressure control, and smoking quit rates were similar across study groups. Patients on intervention teams were more likely to receive NRT (OR = 1.4; 95% CI 1.2-1.6) and enroll in the disease management services MOVE! (OR = 1.2; 95% CI 1.1-1.6) and Telehealth (OR = 1.7, 95% CI 1.4-2.1) than patients on control teams. CONCLUSIONS: Panel Management support for primary care teams improved process, but not outcome variables among veterans with hypertension and smoking. Incorporating PMAs into teams was feasible and highly valued by the clinical staff, but clinical impact may require a longer intervention.

Unannounced standardized patients: a promising method of assessing patient-centered care in your health care system.

BACKGROUND: While unannounced standardized patients (USPs) have been used to assess physicians' clinical skills in the ambulatory setting, they can also provide valuable information on patients' experience of the health care setting beyond the physician encounter. This paper explores the use of USPs as a methodology for evaluating patient-centered care in the health care system. METHODS: USPs were trained to complete a behaviorally-anchored assessment of core dimensions of patient-centered care delivered within the clinical microsystem, including: 1) Medical assistants' safe practices, quality of care, and responsiveness to patients; 2) ease of clinic navigation; and 3) the patient-centeredness of care provided by the physician. Descriptive data is provided on these three levels of patient-centeredness within the targeted clinical microsystem. Chi-square analyses were used to signal whether variations by teams within the clinical microsystem were likely to be due to chance or might reflect true differences in patient-centeredness of specific teams. RESULTS: Sixty USP visits to 11 Primary Care teams were performed over an eight-month period (mean 5 visits/team; range 2-8). No medical assistants reported detecting an USP during the study period. USPs found the clinic easy to navigate and that teams were functioning well in 60% of visits. In 30% to 47% of visits, the physicians could have been more patient-centered. Medical assistants' patient safety measures were poor: patient identity was confirmed in only 5% of visits and no USPs observed medical assistants wash their hands. Quality of care was relatively high for vital signs (e.g. blood pressure, weight and height), but low for depression screening, occurring in only 15% of visits. In most visits, medical assistants greeted the patient in a timely fashion but took time to fully explain matters in less than half of the visits and rarely introduced themselves. Physicians tried to help patients navigate the system in 62% of visits. CONCLUSIONS: USP assessment captured actionable, critical, behaviorally-specific information on team and system performance in an urban community clinic. This methodology provides unique insight into the patient-centeredness and quality of care in medical settings.

Improving adherence to national recommendations for zoster vaccination through simple interventions.

OBJECTIVE: In 2011, 15.8% of eligible patients in the United States were vaccinated against herpes zoster (HZ). To increase the usage of the HZ vaccine by studying physicians' knowledge, attitudes, practices, and perceived obstacles after interventions to overcome barriers. METHODS: General internal medicine physicians were surveyed with a cross-sectional internet survey from October to December 2011 before interventions to increase the use of the HZ vaccine and 1 year later. Interventions included education, increasing availability at the medical center pharmacy, and electronic medical record reminders. Outcome measures included changes in knowledge, attitudes, and practices, and perceived barriers. McNemar chi-square tests were used to compare the changes from the baseline survey for physicians who completed the follow-up survey. RESULTS: Response rate for the baseline study was 33.5% (89/266) and for the follow-up was 29.8% (75/252). Fifty-five completed both surveys. There was a decrease from 57% at baseline to 40% at follow-up in the proportion of physicians who reported that less than 10% of their patients were vaccinated. They were more likely to know the HZ annual incidence (30% baseline; 70% follow-up; P=0.02), and report having educational information for physicians (7% baseline; 27% follow-up; P=0.003). The top helpful intervention was nursing administration of the vaccine. Average monthly HZ vaccine usage in the affiliated outpatient pharmacy increased in 10 months between surveys by 156% compared with the 3 months before the baseline survey. CONCLUSIONS: Interventions implemented during the study led to an increase in physicians' basic knowledge of the HZ vaccine and an increase in usage at the affiliated pharmacy.

Scaffolding the Transition to Residency: A Qualitative Study of Coach and Resident Perspectives.

PURPOSE: This study explores coaching during transition from medical school to residency through the perspectives of residents and faculty coaches participating in a coaching program from residency match through the first year of residency. METHOD: From January to September 2020, 15 faculty coaches in internal medicine, obstetrics and gynecology, emergency medicine, orthopedics, and pathology participated in a synchronous, in-person coaching training course. All 94 postgraduate year 1 residents in these 5 training programs participated. Between November 2021 and March 2022, focus groups were held with interns from all residency programs participating in the program. Interviews were conducted with faculty coaches in February 2022. Faculty and residents discussed their experiences with and perceptions of coaching. De-identified transcripts were coded, and researchers organized these codes into broader categories, generated cross-cutting themes from the concepts described in both cohorts, and proposed a model for the potential of coaching to support the transition to residency. Descriptive themes were constructed and analytic themes developed by identifying concepts that crossed the data sets. RESULTS: Seven focus groups were held with 39 residents (42%). Residents discussed the goals of a coaching program, coach attributes, program factors, resident attributes, and the role of the coach. Coaches focused on productivity of coaching, coaching skills and approach, professional development, and scaffolding the coaching experience. Three analytic themes were created: (1) coaching as creating an explicit curriculum for growth through the transition to residency, (2) factors contributing to successful coaching, and (3) ways in which these factors confront graduate medical education norms. CONCLUSIONS: Learner and faculty perspectives on coaching through the transition to residency reveal the potential for coaching to make an explicit and modifiable curriculum for professional growth and development. Creating structures for coaching in graduate medical education may allow for individualized professional development, improved mindset, self-awareness, and self-directed learning.

Validity evidence for the clinical communication skills assessment tool (CCSAT) from 9 years of implementation in a high stakes medical student OSCE.

OBJECTIVES: Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS: We describe the implementation of the CCSAT including information on the underlying model, the tool's items, domains, scales and scoring, and its role in curriculum. Internal structure is explored through item, internal consistency, and confirmatory factor analyses. Evidence for CCSAT validity is synthesized within prevailing frameworks (Messick12 and Kane13) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS: Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS: This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.

Practice Makes Perfect: Objective Structured Clinical Examinations Across the UME-to-GME Continuum Improve Care of Transgender Simulated Patients.

Background Residents lack confidence in caring for transgender individuals. More exposure and practice throughout training is needed. Objective To explore whether and how prior exposure to transgender health skills during medical school impacted competency with these skills during residency. Methods In 2022, all 101 internal medicine residents at New York University Grossman School of Medicine participated in an objective structured clinical examination (OSCE) station as part of their annual formative assessment where they cared for a standardized patient (SP) who identified as transgender. Three SPs who were members of the transgender community were recruited through online and social media forums. Two resident groups (continuum vs noncontinuum) differed in their prior experiences with transgender OSCEs during medical school. We analyzed SPs' ratings of resident performance using checklist data and SP open-ended feedback to compare performance between groups and resident post-OSCE evaluations to understand residents' perceptions of the educational value of the case. Results Residents with prior experience with transgender SPs (continuum) were more frequently recommended by SPs (88% [21 of 24] vs 70% [54 of 77]) to a family member or friend, were all rated professional (100% [24 of 24] vs 94% [72 of 94]) and scored better in pain information-gathering (92% vs 65%, mean summary score) and gender-affirming care skills (67% vs 52%, mean summary score). Noncontinuum residents lacked experience, missed opportunities to ask about gender identity, and needed work on demonstrating comfort and using proper language. Most residents completing a post-OSCE evaluation (80%, 41 of 51) rated the case as "very valuable." Conclusions Spaced practice and feedback through early exposure to transgender OSCEs were valuable for skill acquisition, giving continuum residents a learning advantage compared to noncontinuum residents.

Using Unannounced Standardized Patients to Assess Clinician Telehealth and Communication Skills at an Urban Student Health Center.

PURPOSE: As the COVID-19 pandemic forced most colleges and universities to go online, student health centers rapidly shifted to telehealth platforms without frameworks for virtual care provision. An urban student health center implemented a needs assessment involving unannounced standardized patients (USPs) to evaluate the integration of a new telehealth workflow and clinicians' virtual communication skills. METHODS: From April to May 2021, USPs conducted two video visits with 12 primary care and four women's health clinicians (N = 16 clinicians; 32 visits). Cases included (1) a 21-year-old female presenting for birth control with a positive Patient Health Questionaire-9 and (2) a 21-year-old male, who vapes regularly, with questions regarding safe sex with men. Clinicians were evaluated using a checklist completed by the USP immediately following the visit and a systematic chart review of the electronic health record. RESULTS: USP feedback indicates most clinicians received high ratings for general communication skills but may benefit from educational intervention in several key telemedicine skills. Clinicians struggled with using nonverbal signals to enrich communication (47% well done), acknowledging emotions (34% well done), and using video for information gathering (34% well done). Low rates of standard screenings (e.g., 63% administered the PHQ-2, <50% asked about alcohol use) suggested protocols for in-person care were not easily incorporated into telehealth practices, and clinicians may benefit from enhanced care team support. Performance reports were shared with clinicians and leadership postvisit. DISCUSSION: Results suggest project design and implementation is scalable and feasible for use at other institutions, offering a structured methodology that can improve general student health care.

Developing a toolkit for panel management: improving hypertension and smoking cessation outcomes in primary care at the VA.

BACKGROUND: As primary care practices evolve into medical homes, there is an increasing need for effective models to shift from visit-based to population-based strategies for care. However, most medical teams lack tools and training to manage panels of patients. As part of a study comparing different approaches to panel management at the Manhattan and Brooklyn campuses of the VA New York Harbor Healthcare System, we created a toolkit of strategies that non-clinician panel management assistants (PMAs) can use to enhance panel-wide outcomes in smoking cessation and hypertension. METHODS: We created the toolkit using: 1) literature review and consultation with outside experts, 2) key informant interviews with staff identified using snowball sampling, 3) pilot testing for feasibility and acceptability, and 4) further revision based on a survey of primary care providers and nurses. These steps resulted in progressively refined strategies for the PMAs to support the primary care team. RESULTS: Literature review and expert consultation resulted in an extensive list of potentially useful strategies. Key informant interviews and staff surveys identified several areas of need for assistance, including help to manage the most challenging patients, providing care outside of the visit, connecting patients with existing resources, and providing additional patient education. The strategies identified were then grouped into 5 areas - continuous connection to care, education and connection to clinical resources, targeted behavior change counseling, adherence support, and patients with special needs. CONCLUSIONS: Although panel management is a central aspect of patient-centered medical homes, providers and health care systems have little guidance or evidence as to how teams should accomplish this objective. We created a toolkit to help PMAs support the clinical care team for patients with hypertension or tobacco use. This toolkit development process could readily be adapted to other behaviors or conditions. TRIAL REGISTRATION: ClinicalTrials.gov, NCT01677533.