Understanding medical student paths to communication skills expertise using latent profile analysis.

PURPOSE: To describe patterns of clinical communication skills that inform curriculum enhancement and guide coaching of medical students. MATERIALS AND METHODS: Performance data from 1182 consenting third year medical students in 9 cohorts (2011-2019), on a 17-item Clinical Communication Skills Assessment Tool (CCSAT) completed by trained Standardized Patients as part of an eight case high stakes Comprehensive Clinical Skills Exam (CCSE) were analyzed using latent profile analysis (LPA). Assessment domains included: information gathering (6 items), relationship development (5 items), patient education (3 items), and organization/time management (3 items). LPA clustered learners with similar strength/weakness into profiles based on item response patterns across cases. One-way analysis of variance (ANOVA) assessed for significant differences by profile for CCSAT items. RESULTS: Student performance clustered into six profiles in three groups, high performing (HP1 and HP2-Low Patient Education, 15.7%), average performing (AP1 and AP2-Interrupters, 40.9%), and lower performing profiles (LP1-Non-interrupters and LP2, 43.4%) with adequate model fit estimations and similar distribution in each cohort. We identified 3 CCSAT items that discriminated among learner's skill profiles. CONCLUSION: Clinical communication skill performance profiles provide nuanced, benchmarked guidance for curriculum improvement and tailoring of communication skills coaching.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

PURPOSE: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). METHOD: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified. RESULTS: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. CONCLUSIONS: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Addressing social determinants of health in primary care: a quasi-experimental study using unannounced standardised patients to evaluate the impact of audit/feedback on physicians' rates of identifying and responding to social needs.

BACKGROUND: Although efforts are underway to address social determinants of health (SDOH), little is known about physicians' SDOH practices despite evidence that failing to fully elicit and respond to social needs can compromise patient safety and undermine both the quality and effectiveness of treatment. In particular, interventions designed to enhance response to social needs have not been assessed using actual practice behaviour. In this study, we evaluate the degree to which providing primary care physicians with feedback on their SDOH practice behaviours is associated with increased rates of eliciting and responding to housing and social isolation needs. METHODS: Unannounced standardised patients (USPs), actors trained to consistently portray clinical scenarios, were sent, incognito, to all five primary care teams in an urban, safety-net healthcare system. Scenarios involved common primary care conditions and each included an underlying housing (eg, mould in the apartment, crowding) and social isolation issue and USPs assessed whether the physician fully elicited these needs and if so, whether or not they addressed them. The intervention consisted of providing physicians with audit/feedback reports of their SDOH practices, along with brief written educational material. A prepost comparison group design was used to evaluate the intervention; four teams received the intervention and one team served as a 'proxy' comparison (no intervention). Preintervention (February 2017 to December 2017) rates of screening for and response to the scripted housing and social needs were compared with intervention period (January 2018 to March 2019) rates for both intervention and comparison teams. RESULTS: 108 visits were completed preintervention and 183 during the intervention period. Overall, social needs were not elicited half of the time and fully addressed even less frequently. Rates of identifying the housing issue increased for teams that received audit/feedback reports (46%-60%; p=0.045) and declined for the proxy comparison (61%-42%; p=0.174). Rates of responding to housing needs increased significantly for intervention teams (15%-41%; p=0.004) but not for the comparison team (21%-29%; p=0.663). Social isolation was identified more frequently postintervention (53%) compared with baseline (39%; p=0.041) among the intervention teams but remained unchanged for the comparison team (39% vs 32%; p=0.601). Full exploration of social isolation remained low for both intervention and comparison teams. CONCLUSIONS: Results suggest that physicians may not be consistently screening for or responding to social needs but that receiving feedback on those practices, along with brief targeted education, can improve rates of SDOH screening and response.

Does it get better? An ongoing exploration of physician experiences with and acceptance of telehealth utilization.

INTRODUCTION: COVID-19 forced health systems to rapidly implement telehealth for routine practice, often without sufficient training or standards. We conducted a longitudinal survey of physicians to explore changes in their perceptions of the challenges and benefits of telehealth and identify recommendations for future practice. METHODS: An anonymous online survey was distributed to a cohort of internal medicine physicians in May to June 2020 and March to June 2021. Changes in responses between 2020 and 2021 and by site (private vs. public) were described. These findings, along with those of a thematic analysis of open-ended responses to questions on telehealth experiences, informed a set of recommendations. RESULTS: The survey response rate was 111/391 in 2021 compared to 122/378 in 2020. Fewer physicians reported that telehealth was more difficult than in person with regards to taking a history (49% in 2020, 33% in 2021, p= 0.015), maximizing patient adherence (33% in 2020, 19% in 2021, p = 0.028), and maintaining patient relationships (31%, 25%, p = 0.009) in 2021. Similar proportions of physicians reported continued challenges with building new patient relationships (75%, 77%, p = 0.075), educating patients (40%, 32%, p = 0.393), and working collaboratively with their team (38%, 41%, p = 0.794). Physicians reported increased satisfaction with tele-visits over in-person visits (13%, 27%, p = 0.006) and less worry over doing future tele-visits (45%, 31%, p = 0.027). Physicians' open-ended responses identified recommendations for further improving the design and use of telehealth. DISCUSSION: Results suggest that physician experience with telehealth improved but opportunities for training and improved integration remain. Longitudinal assessment can deepen understanding of the evolution of telehealth care.

"I Don't Trust It": Use of a Routine OSCE to Identify Core Communication Skills Required for Counseling a Vaccine-Hesitant Patient.

BACKGROUND: Vaccine hesitancy is challenging for clinicians and of increasing concern since COVID-19 vaccination rollout began. Standardized patients (SPs) provide an ideal method for assessing resident physicians' current skills, providing opportunity to practice and gain immediate feedback, while also informing evaluation of curriculum and training. As such, we designed and implemented an OSCE station where residents were tasked with engaging and educating a vaccine-hesitant patient. AIM: Describe residents' vaccine counseling practices, core communication and interpersonal skills, and effectiveness in meeting the objectives of the case. Explore how effectiveness in overcoming vaccine hesitancy may be associated with communication and interpersonal skills in order to inform educational efforts. SETTING: Annual OSCE at a simulation center. PARTICIPANTS: 106 internal medicine residents (51% PGY1, 49% PGY2). PROGRAM DESCRIPTION: Residents participated in an annual residency-wide, multi-station OSCE, one of which included a Black, middle-aged, vaccine-hesitant male presenting for a routine video visit. Residents had 10 min to complete the encounter, during which they sought to educate, explore concerns, and make a recommendation. After each encounter, faculty gave residents feedback on their counseling skills and reviewed best practices for effective communication on the topic. SPs completed a behaviorally anchored checklist (30 items across 7 clinical skill domains and 2 measures of trust in the vaccine's safety and resident) which will inform future curriculum. PROGRAM EVALUATION: Fifty-five percent (SD: 43%) of the residents performed well on the vaccine-specific education domain. PGY2 residents scored significantly higher on two of the seven domains compared to PGY1s (patient education/counseling-PGY1: 35% (SD: 36%) vs. PGY2: 52% (SD: 41%), p = 0.044 and activation-PGY1: 37% (SD: 45%) vs. PGY2: 59% (SD: 46%), p = 0.016). In regression analyses, education/counseling and vaccine-specific communication skills were strongly, positively associated with trust in the resident and in the vaccine's safety. A review of qualitative data from the SPs' perspective suggested that low performers did not use patient-centered communication skills. DISCUSSION: This needs assessment suggests that many residents needed in-the-moment feedback, additional education, and vaccine-specific communication practice. Our program plans to reinforce evidence-based practices physicians can implement for vaccine hesitancy through ongoing curriculum, practice, and feedback. This type of needs assessment is replicable at other institutions and can be used, as we have, to ultimately shed light on next steps for programmatic improvement.

The Telemedicine Takeover: Lessons Learned During an Emerging Pandemic.

Introduction: Health care systems rose to the challenges of COVID-19 by creating or expanding telehealth programs to ensure that patients could access care without an in-person appointment. Traditionally, physicians receive limited formal telemedicine training, making preparedness for this transition uneven. To describe challenges to and attitudes toward providing virtual patient care, we distributed a survey to physicians within our system who largely had no formal prior training/experience with telemedicine, but transitioned to routine telemedicine use. Data collected are then used to offer actionable recommendations for health system leaders and medical educators. Materials and Methods: Surveys were distributed to all faculty outpatient general internal medicine physicians working at any New York University Langone Health, New York City Health + Hospitals/Bellevue and Gouverneur, and the VA NY Harbor Health System (n = 378) in mid-2020. Survey items consisted of Likert and open-ended questions related to experience with televisits (13 items) and attitudes toward care (24 items). Results: Telehealth-related challenges varied by site and modality. Primary challenges included establishing a connection from the patient's (98%) or physician's end (84%) and difficulty in the following domains: working with team members (39%), physical examinations (95%), establishing new patient relationships (70%), and taking history (40%), among others. In thematic analysis, significant themes with illustrative qualitative commentary emerged related to technological challenges, new systems issues, and new patient/physician communication experiences. Discussion: Experience differences were rooted in the type of technology employed. Safety-net practices conducted mostly telephonic visits, whereas private outpatient sites utilized video, despite both using identical electronic medical records. As we consider a "new normal" and prolonged community transmission of COVID-19 infection, it is essential to establish telemedicine training, tools, and protocols that meet the needs of both patients and physicians.

Telemedicine Training in the COVID Era: Revamping a Routine OSCE to Prepare Medicine Residents for Virtual Care.

BACKGROUND: During the rapid onset of the pandemic, clinicians transitioned from traditional outpatient practice to virtual modalities for providing routine care to patient panels. Like training programs nationwide, telemedicine training and assessment had not been systematically incorporated into our residency. In response, a scheduled Internal Medicine (IM) Objective Structured Clinical Examination (OSCE) was adapted to a remote modality to become virtual care-focused learning experience for trainees and to provide valuable feedback to educators. METHODS: Standardized Patients (SPs) rated residents on their communication (including information gathering, relationship development and patient education), patient activation and satisfaction, and telemedicine skills. Analyses included a comparison of domain scores for residents who participated in both the 2020 remote and 2019 in-person OSCEs, and a review of written resident comments about the virtual OSCE. RESULTS: During 2020's video visit OSCE (VOSCE), residents (n = 23) excelled at nonverbal communication but struggled with virtual physical exams and information gathering. In debrief, residents expressed substantial interest in more opportunity to practice virtual visit skills going forward. In comparing scores of the virtual care (2020) OSCE with the in-person (2019) version, the small subset of residents who participated in both assessments (n = 9) performed similarly on communication skills, patient satisfaction and activation. Patient education scores were significantly lower during the virtual care OSCE (P = .008). CONCLUSION: Our reformulated OSCE accomplished 3 goals including; (1) physically distancing residents from SPs per COVID regulations, (2) providing residents with the opportunity to practice critical virtual visit skills, and (3) alerting our educators to curricular improvement areas. Our methods are useful for other institutions and have applications to the larger medical education community.

The COVID-19 Army: Experiences From the Deployment of Non-Hospitalist Physician Volunteers During the COVID-19 Pandemic.

OBJECTIVE: New York City was the epicenter of the outbreak of the 2019 coronavirus disease (COVID-19) pandemic in the United States. As a large, quaternary care medical center, NYU Langone Medical Center was one of many New York medical centers that experienced an unprecedented influx of patients during this time. Clinical leadership effectively identified, oriented, and rapidly deployed a "COVID Army," consisting of non-hospitalist physicians, to meet the needs of the patient influx. We share feedback from our providers on our processes and offer specific recommendations for systems experiencing a similar influx in the current and future pandemics. METHODS: To assess the experiences and perceived readiness of these physicians (n = 183), we distributed a 32-item survey between March and June of 2020. Thematic analyses and response rates were examined to develop results. RESULTS: Responses highlighted varying experiences and attitudes of our frontline physicians during an emerging pandemic. Thematic analyses revealed a series of lessons learned, including the need to (1) provide orientations, (2) clarify roles/workflow, (3) balance team workload, (4) keep teams updated on evolving policies, (5) make team members feel valued, and (6) ensure they have necessary tools available. CONCLUSIONS: Lessons from our deployment and assessment are scalable at other institutions.

Gasping for air: measuring patient education and activation skillsets in two clinical assessment contexts.

Objective structured clinical examinations (OSCEs) provide a controlled, simulated setting for competency assessments, while unannounced simulated patients (USPs) measure competency in situ or real-world settings. This exploratory study describes differences in primary care residents' skills when caring for the same simulated patient case in OSCEs versus in a USP encounter. Data reported describe a group of residents (n=20) who were assessed following interaction with the same simulated patient case in two distinct settings: an OSCE and a USP visit at our safety-net clinic from 2009 to 2010. In both scenarios, the simulated patient presented as an asthmatic woman with limited understanding of illness management. Residents were rated through a behaviourally anchored checklist on visit completion. Summary scores (mean % well done) were calculated by domain and compared using paired sample t-tests. Residents performed significantly better with USPs on 7 of 10 items and in two of three aggregate assessment domains (p<0.05). OSCE structure may impede assessment of activation and treatment planning skills, which are better assessed in real-world settings. This exploration of outcomes from our two assessments using the same clinical case lays a foundation for future research on variation in situated performance. Using both assessments during residency will provide a more thorough understanding of learner competency.

Clinical problem solving and social determinants of health: a descriptive study using unannounced standardized patients to directly observe how resident physicians respond to social determinants of health.

Objectives While the need to address patients' social determinants of health (SDoH) is widely recognized, less is known about physicians' actual clinical problem-solving when it comes to SDoH. Do physicians include SDoH in their assessment strategy? Are SDoH incorporated into their diagnostic thinking and if so, do they document as part of their clinical reasoning? And do physicians directly address SDoH in their "solution" (treatment plan)? Methods We used Unannounced Standardized Patients (USPs) to assess internal medicine residents' clinical problem solving in response to a patient with asthma exacerbation and concern that her moldy apartment is contributing to symptoms - a case designed to represent a clear and direct link between a social determinant and patient health. Residents' clinical practices were assessed through a post-visit checklist and systematic chart review. Patterns of clinical problem solving were identified and then explored, in depth, through review of USP comments and history of present illness (HPI) and treatment plan documentation. Results Residents fell into three groups when it came to clinical problem-solving around a housing trigger for asthma: those who failed to ask about housing and therefore did not uncover mold as a potential trigger (neglectors ­ 21%; 14/68); those who asked about housing in negative ways that prevented disclosure and response (negative elicitors ­ 23%, 16/68); and those who elicited and explored the mold issue (full elicitors ­ 56%; 38/68) [corrected]. Of the full elicitors 53% took no further action, 26% only documented the mold; and 21% provided resources/referral. In-depth review of USP comments/explanations and residents' notes (HPI, treatment plan) revealed possible influences on clinical problem solving. Failure to ask about housing was associated with both contextual factors (rushed visit) and interpersonal skills (not fully engaging with patient) and with possible differences in attention ("known" vs. unknown/new triggers, usual symptoms vs. changes, not attending to relocation, etc.,). Use of close-ended questions often made it difficult for the patient to share mold concerns. Negative responses to sharing of housing information led to missing mold entirely or to the patient not realizing that the physician agreed with her concerns about mold. Residents who fully elicited the mold situation but did not take action seemed to either lack knowledge or feel that action on SDoH was outside their realm of responsibility. Those that took direct action to help the patient address mold appeared to be motivated by an enhanced sense of urgency. Conclusions Findings provide unique insight into residents' problem solving processes including external influences (e.g., time, distractions), the role of core communication and interpersonal skills (eliciting information, creating opportunities for patients to voice concerns, sharing clinical thinking with patients), how traditional cognitive biases operate in practice (premature closure, tunneling, and ascertainment bias), and the ways in which beliefs about expectancies and scope of practice may color clinical problem-solving strategies for addressing SDoH.

Development and maintenance of a medical education research registry.

BACKGROUND: Medical Education research suffers from several methodological limitations including too many single institution, small sample-sized studies, limited access to quality data, and insufficient institutional support. Increasing calls for medical education outcome data and quality improvement research have highlighted a critical need for uniformly clean and easily accessible data. Research registries may fill this gap. In 2006, the Research on Medical Education Outcomes (ROMEO) unit of the Program for Medical Innovations and Research (PrMEIR) at New York University's (NYU) Robert I. Grossman School of Medicine established the Database for Research on Academic Medicine (DREAM). DREAM is a database of routinely collected, de-identified undergraduate (UME, medical school leading up to the Medical Doctor degree) and graduate medical education (GME, residency also known as post graduate education leading to eligibility for specialty board certification) outcomes data available, through application, to researchers. Learners are added to our database through annual consent sessions conducted at the start of educational training. Based on experience, we describe our methods in creating and maintaining DREAM to serve as a guide for institutions looking to build a new or scale up their medical education registry. RESULTS: At present, our UME and GME registries have consent rates of 90% (n = 1438/1598) and 76% (n = 1988/2627), respectively, with a combined rate of 81% (n = 3426/4225). 7% (n = 250/3426) of these learners completed both medical school and residency at our institution. DREAM has yielded a total of 61 individual studies conducted by medical education researchers and a total of 45 academic journal publications. CONCLUSION: We have built a community of practice through the building of DREAM and hope, by persisting in this work the full potential of this tool and the community will be realized. While researchers with access to the registry have focused primarily on curricular/ program evaluation, learner competency assessment, and measure validation, we hope to expand the output of the registry to include patient outcomes by linking learner educational and clinical performance across the UME-GME continuum and into independent practice. Future publications will reflect our efforts in reaching this goal and will highlight the long-term impact of our collaborative work.

Igniting activation: Using unannounced standardized patients to measure patient activation in smoking cessation.

INTRODUCTION: Despite a decline, smoking rates have remained high, especially in communities with lower income, education, and limited insurance options. Evidence shows that physician-initiated counseling on smoking cessation is effective and saves lives, and that specific skills are needed to appropriately lead this type of patient-physician communication. Residency is a critical moment for future physicians and may be the optimal time to learn, practice, and refine this skillset. Unannounced Standardized Patients (USPs) have been found to be effective, incognito evaluators of resident practices. METHODS: This study introduced rigorously trained actors (USPs) into two urban, safety-net clinics to assess resident ability to engage, activate, and counsel a pre-contemplative smoker. A complementary chart review assessed appropriate documentation in the patient's electronic health record (EHR) and its relationship to counseling style and prescribing practices. RESULTS: Resident scores (% well done) on patient education and engagement were low (33% and 23%, respectively). Residents who coupled cessation advice with an open discussion style activated their patients more than those who solely advised cessation across all comparable measures. On EHR documentation, residents who accurately documented smoking history were more likely to directly advise their patient to quit smoking when compared to residents who did not document (t(97) = 2.828, p = .006, Cohen's D = 0.56). CONCLUSIONS: Results highlight the need to reinforce training in patient-centered approaches including motivational interviewing, counseling, and shared decision-making. Future research should focus on the effects of smokers in pre-contemplation on physician counseling style and examine the relationship between medical training and provider communication to guide interventions.