Implementing an Experiential Telehealth Training and Needs Assessment for Residents and Faculty at a Veterans Affairs Primary Care Clinic.

Background: The transition to telehealth during the COVID-19 pandemic revealed a lack of preexisting telehealth training for clinicians. As a workplace-based simulation methodology designed to improve virtual clinical skills, announced standardized patients (ASPs) may help meet evolving educational needs to sustain quality telehealth care. Objective: We describe the development and implementation of an ASP program to assess and provide feedback to resident and faculty clinicians in virtual practice, and report on performance, feasibility, and acceptability. Methods: From June 2021 to April 2022, resident and faculty clinicians at a VA primary care clinic participated in a video visit in which an ASP portrayed either a 70-year-old man with hearing loss and hypertension or a 60-year-old man with hypertension and financial stress. Following the visit, ASPs provided verbal feedback and completed a behaviorally anchored checklist to rate telehealth and communication skills, chronic disease management, and use of resources. Domain summary scores were calculated as the mean percentage of "well done" items. Participants completed a feedback survey on their experience. Results: Seventy-six televisits (60 primary care residents [postgraduate year 1-3], 16 internal medicine faculty) were conducted from August 2021 to April 2022. Clinicians performed well in communication skills: information gathering (79%, 60 of 76, well done), relationship development (67%, 51 of 76), education and counseling (71%, 54 of 76), and patient satisfaction (86%, 65 of 76). They performed less well in telemedicine skills (38%, 29 of 76). Participants agreed that the experience was a good use of their time (88%, 67 of 76). Conclusions: An ASP-facilitated training for resident and faculty clinicians assessed telehealth skills and clinical practice and identified areas for intervention. Clinicians responded well to the training and feedback.

"I Don't Trust It": Use of a Routine OSCE to Identify Core Communication Skills Required for Counseling a Vaccine-Hesitant Patient.

BACKGROUND: Vaccine hesitancy is challenging for clinicians and of increasing concern since COVID-19 vaccination rollout began. Standardized patients (SPs) provide an ideal method for assessing resident physicians' current skills, providing opportunity to practice and gain immediate feedback, while also informing evaluation of curriculum and training. As such, we designed and implemented an OSCE station where residents were tasked with engaging and educating a vaccine-hesitant patient. AIM: Describe residents' vaccine counseling practices, core communication and interpersonal skills, and effectiveness in meeting the objectives of the case. Explore how effectiveness in overcoming vaccine hesitancy may be associated with communication and interpersonal skills in order to inform educational efforts. SETTING: Annual OSCE at a simulation center. PARTICIPANTS: 106 internal medicine residents (51% PGY1, 49% PGY2). PROGRAM DESCRIPTION: Residents participated in an annual residency-wide, multi-station OSCE, one of which included a Black, middle-aged, vaccine-hesitant male presenting for a routine video visit. Residents had 10 min to complete the encounter, during which they sought to educate, explore concerns, and make a recommendation. After each encounter, faculty gave residents feedback on their counseling skills and reviewed best practices for effective communication on the topic. SPs completed a behaviorally anchored checklist (30 items across 7 clinical skill domains and 2 measures of trust in the vaccine's safety and resident) which will inform future curriculum. PROGRAM EVALUATION: Fifty-five percent (SD: 43%) of the residents performed well on the vaccine-specific education domain. PGY2 residents scored significantly higher on two of the seven domains compared to PGY1s (patient education/counseling-PGY1: 35% (SD: 36%) vs. PGY2: 52% (SD: 41%), p = 0.044 and activation-PGY1: 37% (SD: 45%) vs. PGY2: 59% (SD: 46%), p = 0.016). In regression analyses, education/counseling and vaccine-specific communication skills were strongly, positively associated with trust in the resident and in the vaccine's safety. A review of qualitative data from the SPs' perspective suggested that low performers did not use patient-centered communication skills. DISCUSSION: This needs assessment suggests that many residents needed in-the-moment feedback, additional education, and vaccine-specific communication practice. Our program plans to reinforce evidence-based practices physicians can implement for vaccine hesitancy through ongoing curriculum, practice, and feedback. This type of needs assessment is replicable at other institutions and can be used, as we have, to ultimately shed light on next steps for programmatic improvement.

The TeleHealth OSCE: Preparing Trainees to Use Telemedicine as a Tool for Transitions of Care.

BACKGROUND: Telemedicine holds promise to bridge the transition of care between inpatient and outpatient settings. Despite this, the unique communication and technical skills required for virtual encounters are not routinely taught or practiced in graduate medical education (GME) programs. OBJECTIVE: To develop an objective structured clinical examination (OSCE) case to assess residents' telemedicine-specific skills and identify potential gaps in our residency program's curriculum. METHODS: As part of a multi-station OSCE in 2019, we developed a case simulating a remote encounter between a resident and a recently discharged standardized patient. We developed an assessment tool comprising specific behaviors anchored to "not done," "partly done," and "well done" descriptors to evaluate core communication and telemedicine-specific skills. RESULTS: Seventy-eight NYU internal medicine residents participated in the case. Evaluations from 100% of participants were obtained. Residents performed well in Information Gathering and Relationship Development domains. A mean 95% (SD 3.3%) and 91% (SD 4.9%) of residents received "well done" evaluations across these domains. A mean 78% (SD 14%) received "well done" within Education/Counseling domain. However, only 46% (SD 45%) received "well done" evaluations within the Telemedicine domain; specific weak areas included performing a virtual physical examination (18% well done) and leveraging video to augment history gathering (17% well done). There were no differences in telemedicine-specific skill evaluations when stratified by training track or postgraduate year. CONCLUSIONS: We simulate a post-discharge virtual encounter and present a novel assessment tool that uncovers telemedicine-specific knowledge gaps in GME trainees.

"I Cannot Take This Any More!": Preparing Interns to Identify and Help a Struggling Colleague.

BACKGROUND: Few programs train residents in recognizing and responding to distressed colleagues at risk for suicide. AIM: To assess interns' ability to identify a struggling colleague, describe resources, and recognize that physicians can and should help colleagues in trouble. SETTING: Residency programs at an academic medical center. PARTICIPANTS: One hundred forty-five interns. PROGRAM DESIGN: An OSCE case was designed to give interns practice and feedback on their skills in recognizing a colleague in distress and recommending the appropriate course of action. Embedded in a patient "sign-out" case, standardized health professionals (SHP) portrayed a resident with depressed mood and an underlying drinking problem. The SHP assessed intern skills in assessing symptoms and directing the resident to seek help. PROGRAM EVALUATION: Interns appreciated the opportunity to practice addressing this situation. Debriefing the case led to productive conversations between faculty and residents on available resources. Interns' skills require further development: while 60% of interns asked about their colleague's emotional state, only one-third screened for depression and just under half explored suicidal ideation. Only 32% directed the colleague to specific resources for his depression (higher among those that checked his emotional state, 54%, or screened for depression, 80%). DISCUSSION: This OSCE case identified varying intern skill levels for identifying and assessing a struggling colleague while also providing experiential learning and supporting a culture of addressing peer wellness.

Faculty feedback that begins with resident self-assessment: motivation is the key to success.

CONTEXT: The seeking and incorporating of feedback are necessary for continuous performance improvement in medicine. We know that beginning feedback conversations with resident self-assessment may reduce some of the tensions experienced by faculty staff. However, we do not fully understand how residents experience feedback that begins with self-assessment, and whether any existing theoretical frameworks can explain their experiences. METHODS: We conducted a constructivist grounded theory study exploring physical medicine and rehabilitation residents' experiences as they engaged in a structured self-assessment and faculty staff feedback programme. Utilising purposive sampling, we conducted 15 individual interviews and analysed verbatim transcripts iteratively. We implemented several procedures to enhance the credibility of the findings and the protection of participants during recruitment, data collection and data analysis. After defining the themes, we reviewed a variety of existing frameworks to determine if any fitted the data. RESULTS: Residents valued self-assessment followed by feedback (SAFF) and had clear ideas of what makes the process useful. Time pressures and poor feedback quality could lead to a process of 'just going through the motions'. Motivation coloured residents' experiences, with more internalised motivation related to a more positive experience. There were no gender- or year of training-related patterns. CONCLUSIONS: Self-determination theory provided the clearest lens for framing our findings and fitted into a conceptual model linking the quality of the SAFF experience and residents' motivational loci. We identified several study limitations including time in the field, evolving characteristics of the SAFF programme and the absence of faculty voices. We believe that by better understanding residents' experiences of SAFF, educators may be able to tailor the feedback process, enhance clinical performance and ultimately improve patient care.

Charting a Key Competency Domain: Understanding Resident Physician Interprofessional Collaboration (IPC) Skills.

BACKGROUND: Interprofessional collaboration (IPC) is essential for quality care. Understanding residents' level of competence is a critical first step to designing targeted curricula and workplace learning activities. In this needs assessment, we measured residents' IPC competence using specifically designed Objective Structured Clinical Exam (OSCE) cases and surveyed residents regarding training needs. METHODS: We developed three cases to capture IPC competence in the context of physician-nurse collaboration. A trained actor played the role of the nurse (Standardized Nurse - SN). The Interprofessional Education Collaborative (IPEC) framework was used to create a ten-item behaviorally anchored IPC performance checklist (scored on a three-point scale: done, partially done, well done) measuring four generic domains: values/ethics; roles/responsibilities; interprofessional communication; and teamwork. Specific skills required for each scenario were also assessed, including teamwork communication (SBAR and CUS) and patient-care-focused tasks. In addition to evaluating IPC skills, the SN assessed communication, history-taking and physical exam skills. IPC scores were computed as percent of items rated well done in each domain (Cronbach's alpha > 0.77). Analyses include item frequencies, comparison of mean domain scores, correlation between IPC and other skills, and content analysis of SN comments and resident training needs. RESULTS: One hundred and seventy-eight residents (of 199 total) completed an IPC case and results are reported for the 162 who participated in our medical education research registry. IPC domain scores were: Roles/responsibilities mean = 37 % well done (SD 37 %); Values/ethics mean = 49 % (SD 40 %); Interprofessional communication mean = 27 % (SD 36 %); Teamwork mean = 47 % (SD 29 %). IPC was not significantly correlated with other core clinical skills. SNs' comments focused on respect and IPC as a distinct skill set. Residents described needs for greater clarification of roles and more workplace-based opportunities structured to support interprofessional education/learning. CONCLUSIONS: The IPC cases and competence checklist are a practical method for conducting needs assessments and evaluating IPC training/curriculum that provides rich and actionable data at both the individual and program levels.

Learning Skills of Professionalism: a Student-Led Professionalism Curriculum.

BACKGROUND: Medical schools must address the fact that students embarking on careers in medicine are idealistic but have a vague understanding of the values and characteristics that define medical professionalism. Traditionally, we have relied primarily on unsystematic role modeling and lectures or seminars on related topics to teach professionalism. METHODS: A committee of students and a faculty advisor created a curriculum, based on a needs assessment of the targeted learners, to raise students' awareness of professional tenets and provide them with the skills to recognize and analyze conflicts between the values of professionalism and the daily pressures of medical school training. The student-run professionalism curriculum begins during medical school orientation and is followed by three student-facilitated case-based workshops over the next two years. All of the workshops involve small group discussions led by trained upperclass student facilitators. The workshops address the application of professional values to both the preclinical and clinical situations and prepare students for self-reflection, self-assessment and peer evaluation. We evaluated students' satisfaction following each workshop and pre/post attitudes for the first workshop. RESULTS: Twenty five upper-class student facilitators were trained in the first year. Student attendance ranged from 80-100% of the class (N=160), the proportion of students who agreed or strongly agreed that the workshops were educationally useful ranged from 60-75% for each workshop. Certain student attitudes improved immediately after the first workshop. These workshops continue annually. CONCLUSIONS: Students have been a driving force behind this curriculum, which is a model for professionalism education. It was accepted by students and, although somewhat controversial, created a level of awareness and discussion regarding professional behavior in medical school that had previously been absent.