A Cautionary Tale: Exploring Stakeholder Emotions During Curricular Change Using Group Concept Mapping.

PhenomenonCurricular change is essential but challenging. Change efforts often struggle and some fail due to well-articulated challenges as well as other barriers less understood. The curricular change literature characterizes the organizational and operational features of successful change yet virtually ignores stakeholder emotions. A deeper understanding of these emotional responses during the change process could enhance participant and organizational well-being and support change success. Approach In 2019, data were collected at one large North American medical school using group concept mapping methodology, an asynchronous mixed methods approach. We sought to generate themes characterizing the emotional responses of faculty, administrative staff, and students across multiple stakeholder groups participating in the new and traditional (legacy) curricula 27 months after curricular change initiation. Participants brainstormed, sorted, and rated statements on emotional responses. Rating participants rated each statement using a Likert scale from 1 (does not resonate) to 4 (very strongly resonates) according to this prompt: "This emotional response resonates with my emotional experience during the curricular change." Multidimensional scaling and hierarchical cluster analyses were used to generate emotional profiles and compare them across stakeholders. Findings Of 335 stakeholders invited, 123 contributed to brainstorming (36.7%), 153 completed rating (45.7%) and 33 completed sorting (9.9%). Participants generated six themes of emotional responses to curricular change: enthusiastic, apprehensive, overwhelmed, missed opportunities, uncertain, and abandoned. The enthusiastic theme overall had the highest mean statement ratings while the abandoned theme had the lowest. Demographic subgroup analysis revealed new curriculum students (Class of 2021) were most enthusiastic while legacy curriculum students (Class of 2020) were more likely to feel abandoned. Overall, faculty and administrative staff were more enthusiastic than students whereas students rated the five other themes higher than faculty and administrative staff. InsightsCurricular change is emotionally taxing. Students in both curricula experienced greater uncertainty, apprehension, sense of missed opportunities, and feeling overwhelmed than did faculty and administrative staff. Legacy curriculum students rated statements in the abandoned cluster highest while new curriculum students rated statements in the enthusiastic cluster highest. Given the ubiquity of curricular change which often includes a legacy cohort, medical schools embarking on this journey must carefully attend to the varied emotional responses of their different stakeholder groups. The very activities recommended by organizational change models used in medical education, such as communicating wins early and often, could alienate legacy students, creating emotional polarization. These findings suggest that tailored communication strategies are necessary during change implementation to optimize success.

On-demand synchronous audio video telemedicine visits are cost effective.

INTRODUCTION: Claims data raises the possibility that on demand telemedicine programs might increase new utilization, offsetting the cost benefits described in some retrospective analyses. We prospectively evaluated the cost of a synchronous audio-video on-demand telemedicine taking into account both what patients would have done instead of the telemedicine visit as well as the care patients received after the visit. MATERIALS AND METHODS: We conducted a prospective observational study of patients who received care from an on-demand telemedicine program. At the time of the visit, we surveyed patients about the alternative care that would have been requested, if they had not done the telemedicine visit. We also obtained information following the visit about what further care was received. Using cost data derived from the literature we performed a sensitivity analysis to determine the cost impact of the on-demand telemedicine visit. RESULTS: There were 650 patients enrolled with a mean age of 37 who were 68% female; 74% had their care concerns resolved on the telemedicine visit; only 16% would have "done nothing" if they had not done the telemedicine visit, representing possible new utilization. Net cost savings per telemedicine visit was calculated to range from $19-$121 per visit. CONCLUSIONS: In our on-demand telemedicine program, we found the majority of health concerns could be resolved in a single consultation and new utilization was infrequent. Synchronous audio-video telemedicine consults resulted in short-term cost savings by diverting patients from more expensive care settings.

Development of the Oncology Opportunity Cost Assessment Tool: Item Generation and Content Validity Testing.

PURPOSE: The purpose of this study was to develop the Oncology Opportunity Cost Assessment Tool (OOCAT), a survey instrument to evaluate the opportunity costs patients experience when seeking medical oncology care. METHODS: Development of the OOCAT involved extensive patient engagement through both focus groups and interviews. First, the study team developed a list of opportunity cost concepts, which included patients' logistical and financial considerations related to seeking care. We conducted focus groups with patients to expand upon this list of concepts, and then developed a set of questions that incorporated all the concepts generated during the focus groups. To refine these questions, we next performed cognitive interviews with another set of patients to ensure content validity and clarity of instrument items, refining the OOCAT iteratively on the basis of feedback. RESULTS: We engaged 23 participants (17 patients and six caregivers) across four focus groups and 17 participants in cognitive interviews. Focus group participants generated 112 concepts, which resulted in an initial OOCAT with 16 questions. Cognitive interviews resulted in modification of 12 questions and addition of two questions (related to coordination of transportation and impact on home responsibilities). The final OOCAT consisted of 18 items examining time requirements for appointments, financial implications of traveling to appointments for the patient and the caregiver, and logistical and quality-of-life challenges associated with traveling for appointments. CONCLUSION: We developed the OOCAT, an instrument designed to evaluate patient-level opportunity costs of seeking medical oncology care. Further studies to validate the OOCAT are underway.

Emergency Medicine Clinician Experiences Addressing Uncertainty in First-Trimester Bleeding.

The purpose of this work is to understand Emergency Department (ED) clinicians' experiences in communicating uncertainty about first-trimester bleeding (FTB) and their need for training on this topic. This cross-sectional study surveyed a national sample of attending physicians and advanced practice providers (APPs). The survey included quantitative and qualitative questions about communicating with patients presenting with FTB. These questions assessed clinicians' frequency encountering challenges, comfort, training, prior experience, and interest in training on the topic. Of 402 respondents, 54% reported that they encountered challenges at least sometimes when discussing FTB with patients where the pregnancy outcome is uncertain. While the majority (84%) were at least somewhat prepared for these conversations from their training, which commonly addressed the diagnostic approach to this scenario, 39% strongly or moderately agreed that they could benefit from training on the topic. Because the majority of ED clinicians identified at least sometimes encountering challenges communicating with pregnant patients about FTB, our study indicates a need exists for more training in this skill.

Development and preliminary validation of a scale to measure patient uncertainty: The "Uncertainty Scale".

Research suggests that patient uncertainty related to experiencing symptoms may drive decisions to seek care. The only validated measure of patient uncertainty assesses uncertainty related to defined illness. In prior work, we engaged patients to describe uncertainty related to symptoms and used findings to develop the 'U-Scale' scale. In this work, we present results from preliminary scale reliability and validity testing. Psychometric testing demonstrated content validity, high internal consistency, and evidence for concurrent validity. Next steps include administration in diverse populations for continued refinement and validation, and exploration of the potential contribution of uncertainty to healthcare utilization.

Patient experience and challenges in group concept mapping for clinical research.

BACKGROUND AND OBJECTIVE: Group concept mapping (GCM) is a research method that engages stakeholders in generating, structuring and representing ideas around a specific topic or question. GCM has been used with patients to answer questions related to health and disease but little is known about the patient experience as a participant in the process. This paper explores the patient experience participating in GCM as assessed with direct observation and surveys of participants. METHODS: This is a secondary analysis performed within a larger study in which 3 GCM iterations were performed to engage patients in identifying patient-important outcomes for diabetes care. Researchers tracked the frequency and type of assistance required by each participant to complete the sorting and rating steps of GCM. In addition, a 17-question patient experience survey was administered over the telephone to the participants after they had completed the GCM process. Survey questions asked about the personal impact of participating in GCM and the ease of various steps of the GCM process. RESULTS: Researchers helped patients 92 times during the 3 GCM iterations, most commonly to address software and computer literacy issues, but also with the sorting phase itself. Of the 52 GCM participants, 40 completed the post-GCM survey. Respondents averaged 56 years of age, were 50% female and had an average hemoglobin A1c of 9.1%. Ninety-two percent (n = 37) of respondents felt that they had contributed something important to this research project and 90% (n = 36) agreed or strongly agreed that their efforts would help others with diabetes. Respondents reported that the brainstorming session was less difficult when compared with sorting and rating of statements. DISCUSSION: Our results suggest that patients find value in participating in GCM. Patients reported less comfort with the sorting step of GCM when compared with brainstorming, an observation that correlates with our observations from the GCM sessions. Researchers should consider using paper sorting methods and objective measures of sorting quality when using GCM in patient-engaged research to improve the patient experience and concept map quality.

Identifying Emergency Department Symptom-Based Diagnoses with the Unified Medical Language System.

INTRODUCTION: Many patients who are discharged from the emergency department (ED) with a symptom-based discharge diagnosis (SBD) have post-discharge challenges related to lack of a definitive discharge diagnosis and follow-up plan. There is no well-defined method for identifying patients with a SBD without individual chart review. We describe a method for automated identification of SBDs from ICD-10 codes using the Unified Medical Language System (UMLS) Metathesaurus. METHODS: We mapped discharge diagnosis, with use of ICD-10 codes from a one-month period of ED discharges at an urban, academic ED to UMLS concepts and semantic types. Two physician reviewers independently manually identified all discharge diagnoses consistent with SBDs. We calculated inter-rater reliability for manual review and the sensitivity and specificity for our automated process for identifying SBDs against this "gold standard." RESULTS: We identified 3642 ED discharges with 1382 unique discharge diagnoses that corresponded to 875 unique ICD-10 codes and 10 UMLS semantic types. Over one third (37.5%, n = 1367) of ED discharges were assigned codes that mapped to the "Sign or Symptom" semantic type. Inter-rater reliability for manual review of SBDs was very good (0.87). Sensitivity and specificity of our automated process for identifying encounters with SBDs were 84.7% and 96.3%, respectively. CONCLUSION: Use of our automated process to identify ICD-10 codes that classify into the UMLS "Sign or Symptom" semantic type identified the majority of patients with a SBD. While this method needs refinement to increase sensitivity of capture, it has potential to automate an otherwise highly time-consuming process. This novel use of informatics methods can facilitate future research specific to patients with SBDs.