Finding the Needle in the Haystack: Can Natural Language Processing of Students' Evaluations of Teachers Identify Teaching Concerns?

BACKGROUND: Institutions rely on student evaluations of teaching (SET) to ascertain teaching quality. Manual review of narrative comments can identify faculty with teaching concerns but can be resource and time-intensive. AIM: To determine if natural language processing (NLP) of SET comments completed by learners on clinical rotations can identify teaching quality concerns. SETTING AND PARTICIPANTS: Single institution retrospective cohort analysis of SET (n = 11,850) from clinical rotations between July 1, 2017, and June 30, 2018. PROGRAM DESCRIPTION: The performance of three NLP dictionaries created by the research team was compared to an off-the-shelf Sentiment Dictionary. PROGRAM EVALUATION: The Expert Dictionary had an accuracy of 0.90, a precision of 0.62, and a recall of 0.50. The Qualifier Dictionary had lower accuracy (0.65) and precision (0.16) but similar recall (0.67). The Text Mining Dictionary had an accuracy of 0.78 and a recall of 0.24. The Sentiment plus Qualifier Dictionary had good accuracy (0.86) and recall (0.77) with a precision of 0.37. DISCUSSION: NLP methods can identify teaching quality concerns with good accuracy and reasonable recall, but relatively low precision. An existing, free, NLP sentiment analysis dictionary can perform nearly as well as dictionaries requiring expert coding or manual creation.

Potential clinical utility of 68Ga-DOTATATE PET/CT for detection and response assessment in cardiac sarcoidosis.

BACKGROUND: Somatostatin receptor is expressed in sarcoid granulomas, and preliminary clinical studies have shown that myocardial sarcoidosis can be identified on somatostatin receptor-targeted PET. We examined the potential clinical use of 68Ga-DOTATATE PET/CT for diagnosis and response assessment in cardiac sarcoidosis compared to 18F-FDG PET/CT. METHODS: Eleven cardiac sarcoidosis patients with 18F-FDG PET/CT were prospectively enrolled for cardiac 68Ga-DOTATATE PET/CT. The two PET/CT studies were interpreted independently and were compared for patient-level and segment-level concordance, as well as for the degree of radiotracer uptake. Follow-up 68Ga-DOTATATE PET/CT was performed in eight patients. RESULTS: Patient-level concordance was 91%: ten patients had multifocal DOTATATE uptake (active cardiac sarcoidosis) and one patient showed diffuse DOTATATE uptake. Segment-level agreement was 77.1% (Kappa 0.53 ± 0.07). The SUVmax-to-blood pool ratio was lower on 68Ga-DOTATATE PET/CT (3.2 ± 0.6 vs. 4.9 ± 1.5, P = 0.006 on paired t test). Follow-up 68Ga-DOTATATE PET/CT showed one case of complete response and one case of partial response, while 18F-FDG PET/CT showed four cases of response, including three with complete response. CONCLUSION: Compared to 18F-FDG PET/CT, 68Ga-DOTATATE PET/CT can identify active cardiac sarcoidosis with high patient-level concordance, but with moderate segment-level concordance, low signal-to-background ratio, and underestimation of treatment response.

The Context of "Confidence": Analyzing the Term Confidence in Resident Evaluations.

BACKGROUND: Despite similar performance metrics, women medical trainees routinely self-assess their own skills lower than men. The phenomenon of a "confidence gap" between genders, where women report lower self-confidence independent of actual ability or competency, may have an important interaction with gender differences in assessment. Identifying whether there are gender-based differences in how confidence is mentioned in written evaluations is a necessary step to understand the interaction between evaluation and the gender-based confidence gap. OBJECTIVE: To analyze faculty evaluations of internal medicine (IM) residents for gender-based patterns in the use of iterations of "confidence." DESIGN: We performed a retrospective cohort study of all inpatient faculty evaluations of University of Pennsylvania IM residents from 2018 to 2021. We performed n-gram text-mining to identify evaluations containing the terms "confident," "confidence," or "confidently." We performed univariable and multivariable logistic regression to determine the association between resident gender and references to confidence (including comments reflecting too little confidence), adjusting for faculty gender, post-graduate year (PGY), numeric rating, and service. SUBJECTS: University of Pennsylvania IM residents from 2018 to 2021. KEY RESULTS: There were 5416 evaluations of IM residents (165 women [51%], 156 men [49%]) submitted by 356 faculty members (149 women [51%]), of which 7.1 % (n=356) contained references to confidence. There was a significant positive association between the mention of confidence and women resident gender (OR 1.54, CI 1.23-1.92; p<0.001), which persisted after adjustment for faculty gender, numeric rating, and PGY level. Eighty evaluations of the cohort explicitly mentioned the resident having "too little confidence," which was also associated with women resident gender (OR 1.66, CI 1.05-2.62; p=0.031). CONCLUSION: Narrative evaluations of women residents were more likely to contain references to confidence, after adjustment for numerical score, PGY level, and faculty gender, which may perpetuate the gender-based confidence gap, introduce bias, and ultimately impact professional identity development.

Risk Factors, Etiologies, and Screening Tools for Sepsis in Pregnant Women: A Multicenter Case-Control Study.

BACKGROUND: Given the significant morbidity and mortality of maternal sepsis, early identification is key to improve outcomes. This study aims to evaluate the performance characteristics of the systemic inflammatory response syndrome (SIRS), quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA), and maternal early warning (MEW) criteria for identifying cases of impending sepsis in parturients. The secondary objective of this study is to identify etiologies and risk factors for maternal sepsis and to assess timing of antibiotics in patients diagnosed with sepsis. METHODS: Validated maternal sepsis cases during the delivery hospitalization from 1995 to 2012 were retrospectively identified at 7 academic medical centers in the United States and Israel. Control patients were matched by date of delivery in a 1:4 ratio. The sensitivity and specificity of SIRS, qSOFA, and MEW criteria for identifying sepsis were calculated. Data including potential risk factors, vital signs, laboratory values, and clinical management were collected for cases and controls. RESULTS: Eighty-two sepsis cases during the delivery hospitalization were identified and matched to 328 controls. The most common causes of sepsis were the following: chorioamnionitis 20 (24.4%), endometritis 19 (23.2%), and pneumonia 9 (11.0%). Escherichia coli 12 (14.6%), other Gram-negative rods 8 (9.8%), and group A Streptococcus 6 (7.3%) were the most commonly found pathogens. The sensitivities and specificities for meeting criteria for screening tools were as follows: (1) SIRS (0.93, 0.63); (2) qSOFA (0.50, 0.95); and (3) MEW criteria for identifying sepsis (0.82, 0.87). Of 82 women with sepsis, 10 (12.2%) died. The mortality rate for those who received antibiotics within 1 hour of diagnosis was 8.3%. The mortality rate was 20% for the patients who received antibiotics after >1 hour. CONCLUSIONS: Chorioamnionitis and endometritis were the most common causes of sepsis, together accounting for about half of cases. Notable differences were observed in the sensitivity and specificity of sepsis screening tools with the highest to lowest sensitivity being SIRS, MEW, and qSOFA criteria, and the highest to lowest specificity being qSOFA, MEW, and SIRS. Mortality was doubled in the cohort of patients who received antibiotics after >1 hour. Clinicians need to be vigilant to identify cases of peripartum sepsis early in its course and prioritize timely antibiotic therapy.

Treatment patterns and short-term outcomes in ischemic stroke in pregnancy or postpartum period.

BACKGROUND: Stroke, which is a rare but devastating event during pregnancy, occurs in 34 of every 100,000 deliveries; obstetricians are often the first providers to be contacted by symptomatic patients. At least one-half of pregnancy-related strokes are likely to be of the ischemic stroke subtype. Most pregnant or newly postpartum women with ischemic stroke do not receive acute stroke reperfusion therapy, although this is the recommended treatment for adults. Little is known about these therapies in pregnant or postpartum women because pregnancy has been an exclusion criterion for all reperfusion trials. Until recently, pregnancy and obstetric delivery were specifically identified as warnings to intravenous alteplase tissue plasminogen activator in Federal Drug Administration labeling. OBJECTIVE: The primary study objective was to compare the characteristics and outcomes of pregnant or postpartum vs nonpregnant women with ischemic stroke who received acute reperfusion therapy. STUDY DESIGN: Pregnant or postpartum (<6 weeks; n = 338) and nonpregnant (n = 24,303) women 18-44 years old with ischemic stroke from 1991 hospitals that participated in the American Heart Association's Get With the Guidelines-Stroke Registry from 2008-2013 were identified by medical history or International Classification of Diseases, Ninth Revision, codes. Acute stroke reperfusion therapy was defined as intravenous tissue plasminogen activator, catheter-based thrombolysis, or thrombectomy or any combination thereof. A sensitivity analysis was done on patients who received intravenous tissue plasminogen activator monotherapy only. Chi-square tests were used for categoric variables, and Wilcoxon Rank-Sum was used for continuous variables. Conditional logistic regression was used to assess the association of pregnancy with short-term outcomes. RESULTS: Baseline characteristics of the pregnant or postpartum vs nonpregnant women with ischemic stroke revealed a younger group who, despite greater stroke severity, were less likely to have a history of hypertension or to arrive via emergency medical services. There were similar rates of acute stroke reperfusion therapy in the pregnant or postpartum vs nonpregnant women (11.8% vs 10.5%; P = .42). Pregnant or postpartum women were less likely to receive intravenous tissue plasminogen activator monotherapy (4.4% vs 7.9%; P = .03), primarily because of pregnancy and recent surgery. There was a trend toward increased symptomatic intracranial hemorrhage in the pregnant or postpartum patients who were treated with tissue plasminogen activator, yet no cases of major systemic bleeding or in-hospital death occurred, and there were similar rates of discharge to home. Data on the timing of pregnancy, which were available in 145 of 338 cases, showed that 44.8% of pregnancy-related strokes were antepartum, that 2.8% occurred during delivery, and that 52.4% were during the postpartum period. CONCLUSIONS: Using data from the Get With the Guidelines-Stroke Registry to assemble the largest cohort of pregnant or postpartum ischemic stroke patients who had been treated with reperfusion therapy, we observed that pregnant or postpartum women had similarly favorable short-term outcomes and equal rates of total reperfusion therapy to nonpregnant women, despite lower rates of intravenous tissue plasminogen activator use. Future studies should identify the characteristics of pregnant and postpartum ischemic stroke patients who are most likely to safely benefit from reperfusion therapy.

Opioid abuse and dependence during pregnancy: temporal trends and obstetrical outcomes.

BACKGROUND: The authors investigated nationwide trends in opioid abuse or dependence during pregnancy and assessed the impact on maternal and obstetrical outcomes in the United States. METHODS: Hospitalizations for delivery were extracted from the Nationwide Inpatient Sample from 1998 to 2011. Temporal trends were assessed and logistic regression was used to examine the associations between maternal opioid abuse or dependence and obstetrical outcomes adjusting for relevant confounders. RESULTS: The prevalence of opioid abuse or dependence during pregnancy increased from 0.17% (1998) to 0.39% (2011) for an increase of 127%. Deliveries associated with maternal opioid abuse or dependence compared with those without opioid abuse or dependence were associated with an increased odds of maternal death during hospitalization (adjusted odds ratio [aOR], 4.6; 95% CI, 1.8 to 12.1, crude incidence 0.03 vs. 0.006%), cardiac arrest (aOR, 3.6; 95% CI, 1.4 to 9.1; 0.04 vs. 0.01%), intrauterine growth restriction (aOR, 2.7; 95% CI, 2.4 to 2.9; 6.8 vs. 2.1%), placental abruption (aOR, 2.4; 95% CI, 2.1 to 2.6; 3.8 vs. 1.1%), length of stay more than 7 days (aOR, 2.2; 95% CI, 2.0 to 2.5; 3.0 vs. 1.2%), preterm labor (aOR, 2.1; 95% CI, 2.0 to 2.3; 17.3 vs. 7.4%), oligohydramnios (aOR, 1.7; 95% CI, 1.6 to 1.9; 4.5 vs. 2.8%), transfusion (aOR, 1.7; 95% CI, 1.5 to 1.9; 2.0 vs. 1.0%), stillbirth (aOR, 1.5; 95% CI, 1.3 to 1.8; 1.2 vs. 0.6%), premature rupture of membranes (aOR, 1.4; 95% CI, 1.3 to 1.6; 5.7 vs. 3.8%), and cesarean delivery (aOR, 1.2; 95% CI, 1.1 to 1.3; 36.3 vs. 33.1%). CONCLUSIONS: Opioid abuse or dependence during pregnancy is associated with considerable obstetrical morbidity and mortality, and its prevalence is dramatically increasing in the United States. Identifying preventive strategies and therapeutic interventions in pregnant women who abuse drugs are important priorities for clinicians and scientists.

A virtual simulation-based clinical skills course.

BACKGROUND: Advancements in technology continue to transform the landscape of medical education. Virtual reality (VR) and remote simulation represent unique approaches to teaching students clinical skills through experiential learning. APPROACH: We developed, implemented and evaluated a virtual simulation-based course modelled on Kolb's Learning Cycle. A 5-day, 15-h virtual clerkship clinical skills curriculum designed to help students recognise a patient requiring urgent or emergent care was piloted in May 2020. VR and remote simulations facilitated concrete experiences, reflective observation, concept formation, and active experimentation. Curricular evaluation included analysis of usage, performance, self-assessment, and perceptions of educational value. EVALUATION: One hundred and fifty-six students completed 436 VR sessions (median 3 per student). Students repeated virtual cases, on average, 75% of the time, demonstrating improvement in average performance scores from 59% (attempt 1) to 72% (attempt 2). Post-course evaluations, completed by 109 (69.9%) students, demonstrated significant improvements in mean scores in all domains of a self-assessment based on emergent care skills. IMPLICATIONS: We developed and implemented a virtual clinical skills course that allowed students to progress through all four stages of Kolb's Learning Cycle. VR and remote simulations represent an opportunity for educators to reimagine and expand opportunities for experiential learning. The number of students accommodated, total virtual simulations completed, and positive student feedback suggests that this may be a feasible, acceptable, and scalable method for increasing opportunities for progression through Kolb's Learning Cycle. Further investigation into impact on educational outcomes is needed.

Insole Pressure Sensors to Assess Post-Stroke Gait.

OBJECTIVE: To confirm that the simplified insole does not affect the gait speed and to identify objective sensor-based gait parameters that correlate strongly with existing clinical gait assessment scales. METHODS: Ten participants with gait impairment due to hemiplegic stroke were enrolled in this study. Pairs of insoles with four pressure sensors on each side were manufactured and placed in each shoe. Data were extracted during the 10-Meter Walk Test. Several sensor-derived parameters (for example stance time, heel_on-to-toe_peak time, and toe_peak pressure) were calculated and correlated with gait speed and lower extremity Fugl-Meyer (F-M) score. RESULTS: The insole pressure sensor did not affect gait, as indicated by a strong correlation (ρ=0.988) and high agreement (ICC=0.924) between the gait speeds with and without the insole. The parameters that correlated most strongly with highest ß coefficients against the clinical measures were stance time of the non-hemiplegic leg (ß=-0.87 with F-M and ß=-0.95 with gait speed) and heel_on-to-toe_peak time of the non-hemiplegic leg (ß=-0.86 with F-M and -0.94 with gait speed). CONCLUSION: Stance time of the non-hemiparetic leg correlates most strongly with clinical measures and can be assessed using a non-obtrusive insole pressure sensor that does not affect gait function. These results suggest that an insole pressure sensor, which is applicable in a home environment, may be useful as a clinical endpoint in post-stroke gait therapy trials.

Relationship of Ketosis With Myocardial Glucose Uptake Among Patients Undergoing FDG PET/CT for Evaluation of Cardiac Sarcoidosis.

BACKGROUND: Fluorine-18 fluorodeoxyglucose (FDG) with positron emission tomography (PET) is the standard for detecting myocardial inflammation in cardiac sarcoidosis, requiring preparation with the ketogenic diet (KD) to achieve myocardial glucose suppression. Despite this, incomplete myocardial glucose suppression remains a significant issue, and strategies to reduce myocardial glucose uptake (MGU) and identify incomplete myocardial glucose suppression are required. This study sought to understand the relationship between point-of-care beta-hydroxybutyrate (BHB) and different patterns of MGU and between KD and fasting duration with MGU in patients undergoing evaluation for cardiac sarcoidosis. METHODS: We prospectively included 471 outpatients who underwent FDG-PET for cardiac sarcoidosis evaluation, followed the KD for 1 (n=100), 2 (n=29), and ≥3 days (n=342), fasted for at least 12 hours, and had BHB levels measured immediately before FDG injection. Images were classified as (1) no MGU (negative), (2) focal/multifocal (positive), (3) diffuse (nondiagnostic), or (4) nonspecific uptake (NS-MGU). RESULTS: Cardiac FDG-PET scans were interpreted as the following: 376 (79.83%) negative; 61 (12.95%) positive; 14 (2.97%) diffuse; and 20 (4.25%) NS-MGU. There was a strong negative relationship between BHB levels and MGU (P<0.0001). BHB levels increased significantly with KD duration (P<0.0001) and fasting time (P=0.0067). The combined rate of diffuse, NS-MGU, and positive scans (34%, 28%, 16%) decreased inversely with KD duration (1, 2, and ≥3 days, respectively). However, MGU was not different across different fasting times (P=0.6). Blood glucose levels were not associated with MGU (P=0.17) and only weakly associated with BHB levels (R2=0.03; P<0.001). CONCLUSIONS: We observed a strong inverse relationship between ketosis and patterns of MGU. Longer KD and fasting durations are associated with higher ketosis. However, only KD duration was associated with lower rates of MGU. Measurement of BHB levels before FDG-PET using point-of-care testing is feasible and may facilitate the management of patients referred for myocardial inflammation.

Better Together: Development and Implementation of Fellow Group Evaluations of Faculty.

Background: High-quality trainee evaluations of faculty are essential for meaningful faculty development and for improving the clinical learning environment. However, concerns about anonymity can limit usefulness of trainee evaluations, particularly in smaller programs, such as subspecialty fellowships. Objective: To develop and implement a fellow-driven group evaluation process to enhance trainee confidentiality and generate high-quality feedback for pulmonary and critical care medicine faculty. Methods: A novel process was developed for faculty evaluation and feedback consisting of quarterly, structured, fellow-led group evaluation sessions focused on collecting confidential, behaviorally oriented, actionable feedback for faculty. Upper-year fellow moderators utilized a standard format to structure discussion, generating strengths and areas for growth for each faculty member while explicitly asking for input from fellows with divergent perspectives. Moderators compiled anonymized session notes for the program director, who delivered feedback to individual faculty. After the first six sessions, an electronic survey was distributed to assess fellow perceptions of the group evaluation model. Results: Thirty-seven faculty members were evaluated in 11 group sessions over 42 months. Fellows rated group-generated feedback as more confidential, more specific, more accurate, more efficient, more actionable, and less biased when compared with individual written evaluations (P < 0.01 for all categories). Conclusion: The authors successfully developed and implemented a process for fellow-led group evaluation of faculty, designed to facilitate fellow confidentiality and enrich the quality of feedback. Fellows preferred the group evaluation process and perceived group-generated feedback more favorably compared with individual written evaluations.

Muscle-driven simulations and experimental data of cycling.

Muscle-driven simulations have provided valuable insights in studies of walking and running, but a set of freely available simulations and corresponding experimental data for cycling do not exist. The aim of this work was to develop a set of muscle-driven simulations of cycling and to validate them by comparison with experimental data. We used direct collocation to generate simulations of 16 participants cycling over a range of powers (40-216 W) and cadences (75-99 RPM) using two optimization objectives: a baseline objective that minimized muscle effort and a second objective that additionally minimized tibiofemoral joint forces. We tested the accuracy of the simulations by comparing the timing of active muscle forces in our baseline simulation to timing in experimental electromyography data. Adding a term in the objective function to minimize tibiofemoral forces preserved cycling power and kinematics, improved similarity between active muscle force timing and experimental electromyography, and decreased tibiofemoral joint reaction forces, which better matched previously reported in vivo measurements. The musculoskeletal models, muscle-driven simulations, simulation software, and experimental data are freely shared at https://simtk.org/projects/cycling_sim for others to reproduce these results and build upon this research.

Natural Language Processing of Learners' Evaluations of Attendings to Identify Professionalism Lapses.

Unprofessional faculty behaviors negatively impact the well-being of trainees yet are infrequently reported through established reporting systems. Manual review of narrative faculty evaluations provides an additional avenue for identifying unprofessional behavior but is time- and resource-intensive, and therefore of limited value for identifying and remediating faculty with professionalism concerns. Natural language processing (NLP) techniques may provide a mechanism for streamlining manual review processes to identify faculty professionalism lapses. In this retrospective cohort study of 15,432 narrative evaluations of medical faculty by medical trainees, we identified professionalism lapses using automated analysis of the text of faculty evaluations. We used multiple NLP approaches to develop and validate several classification models, which were evaluated primarily based on the positive predictive value (PPV) and secondarily by their calibration. A NLP-model using sentiment analysis (quantifying subjectivity of the text) in combination with key words (using the ensemble technique) had the best performance overall with a PPV of 49% (CI 38%-59%). These findings highlight how NLP can be used to screen narrative evaluations of faculty to identify unprofessional faculty behaviors. Incorporation of NLP into faculty review workflows enables a more focused manual review of comments, providing a supplemental mechanism to identify faculty professionalism lapses.

Education for Tobacco Use Disorder Treatment: Current State, Evidence, and Unmet Needs.

Background: Tobacco use is undertreated in the medical setting. One driver may be inadequate tobacco use disorder treatment (TUDT) training for clinicians in specialties treating tobacco-dependent patients. Objective: We sought to evaluate the current state of TUDT training for diverse professionals and how these skills are assessed in credentialing exams. Methods: We performed a focused review of current educational practices, evidence-based strategies, and accreditation exam contents focused on TUDT. Results: Among medical students, participants in reviewed studies reported anywhere from 45 minutes to 3 hours of TUDT training throughout their 4-year programs, most often in the form of didactic sessions. Similarly, little TUDT training was reported at the post-graduate (residency, fellowship, continuing medical education) levels, and reported training was typically delivered as time-based (expected hours of instruction) rather than competency-based (demonstration of mastery) learning. Multiple studies evaluated effective TUDT curricula at varied stages of training. More effective curricula incorporated longitudinal sessions and active learning, such as standardized patient encounters or proctored patient visits. Knowledge of TUDT is minimally evaluated on certification exams. For example, the American Board of Internal Medicine blueprint lists TUDT as <2% of one subtopic on both the internal medicine and pulmonary exams. Conclusion: TUDT training for most clinicians is minimal, does not assess competency, and is minimally evaluated on certification exams. Effective, evidence-based TUDT training incorporating active learning should be integrated into medical education at all levels, with attention paid to inclusion on subsequent certifying exams.

Diagnostic Reasoning of Resident Physicians in the Age of Clinical Pathways.

Background: Development of skills in diagnostic reasoning is paramount to the transition from novice to expert clinicians. Efforts to standardize approaches to diagnosis and treatment using clinical pathways are increasingly common. The effects of implementing pathways into systems of care during diagnostic education and practice among pediatric residents are not well described. Objective: To characterize pediatric residents' perceptions of the tradeoffs between clinical pathway use and diagnostic reasoning. Methods: We conducted a qualitative study from May to December 2019. Senior pediatric residents from a high-volume general pediatric inpatient service at an academic hospital participated in semi-structured interviews. We utilized a basic interpretive qualitative approach informed by a dual process diagnostic reasoning framework. Results: Nine residents recruited via email were interviewed. Residents reported using pathways when admitting patients and during teaching rounds. All residents described using pathways primarily as management tools for patients with a predetermined diagnosis, rather than as aids in formulating a diagnosis. As such, pathways primed residents to circumvent crucial steps of deliberate diagnostic reasoning. However, residents relied on bedside assessment to identify when patients are "not quite fitting the mold" of the current pathway diagnosis, facilitating recalibration of the diagnostic process. Conclusions: This study identifies important educational implications at the intersection of residents' cognitive diagnostic processes and use of clinical pathways. We highlight potential challenges clinical pathways pose for skill development in diagnostic reasoning by pediatric residents. We suggest opportunities for educators to leverage clinical pathways as a framework for development of these skills.

Comparative Evaluation of Choose Your Own Adventure and Traditional Linear Case Formats in Radiology Small Group Teaching.

RATIONALE AND OBJECTIVES: We aim to compare Choose Your Own Adventure (CYOA) presentation format with linear case format as educational methods for teaching a radiology small group session to medical students. MATERIALS AND METHODS: A radiology small group session was held for preclinical second-year medical students in the pulmonary course, whereby eight classrooms of students and eight radiology facilitators were each randomized to do either the linear case format or the nonlinear CYOA presentation format. All students in attendance were administered a survey at the end of the session, which assessed students' perceptions using five-point Likert-type questions. The survey also contained a four-question knowledge quiz on chest radiology. The facilitators were administered a qualitative survey as well. Between-group analyses were performed using Student's t-test. RESULTS: Of the 144 students who attended the small group sessions, 143 students completed the survey (99.3%). The CYOA format group reported significantly greater engagement in the cases (4.5 ± 0.7 vs. 3.8 ± 0.7, p < 0.001), satisfaction with the format (4.6 ± 0.6 vs. 3.7 ± 0.9, p < 0.001), and enhancement of clinical decision making skills (4.5 ± 0.6 vs. 3.5 ± 0.9, p < 0.001). The linear format group reported a greater role for the facilitator to add value (4.6 ± 0.5 vs. 4.3 ± 1.1, p = 0.033). There was no significant difference between groups in performance on the knowledge quiz. CONCLUSION: Medical students reported higher satisfaction, engagement, and enhanced clinical decision making skills with the CYOA presentation method compared to linear case format for radiology small group learning.

Effect of Immunosuppressive Therapy and Biopsy Status in Monitoring Therapy Response in Suspected Cardiac Sarcoidosis.

BACKGROUND: Patients with suspected cardiac sarcoidosis frequently undergo fluorodeoxyglucose (FDG)-positron emission tomography (PET)/computed tomography (CT) imaging to assess disease activity at baseline and after treatment initiation. OBJECTIVES: This study investigated the effect of immunosuppressive therapy and biopsy status to achieve complete treatment response (CTR), partial treatment response (PTR), or no response (NR) on myocardial FDG-PET/CT. METHODS: This study analyzed 83 patients with suspected cardiac sarcoidosis (aged 53 ± 1.8 years, 71% were male, 69% were White, 61% had a history of biopsy-confirmed sarcoidosis) who were treatment naive, had evidence of myocardial FDG at baseline, and underwent repeat PET imaging after treatment initiation. CTR was graded visually, and PTR/NR were measured both visually and quantitatively using the total glycolytic activity. Patients were also evaluated for the occurrence of death, sustained ventricular arrhythmias, and heart failure admissions. RESULTS: Overall, 59 patients (71%) achieved CTR/PTR (30%/41%) at follow-up scan (P = 0.04). Total glycolytic activity and visual estimate of PTR/NR had excellent agreement (κ = 0.86 [95% CI: 0.72-0.99]; P < 0.0001). In patients receiving prednisone only, the highest rates of CTR/PTR were observed in patients initiated on moderate or high dose (P < 0.01). In a regression model, moderate prednisone start dose (P = 0.03) was more strongly associated with achieving CTR/PTR than was high prednisone start dose. However, the latter patients were tapered faster between start dose and follow-up scan (P < 0.01). After a median follow-up of 4.7 (IQR: 3.1-7.8) years, patients who were biopsy-proven (vs non-biopsy-proven; P = 0.029) and with preserved left ventricular function (P = 002) were less likely to experience major adverse cardiac events. Outcomes based on treatment response status (CTR vs PTR vs NR; P = 0.23) were not significantly different. CONCLUSIONS: Among patients with suspected sarcoidosis and evidence of myocardial inflammation, treatment response by serial FDG-PET was variable, but a favorable response was more common when using moderate-to-high intensity prednisone dose. Biopsy-proven individuals and those with preserved systolic function were less likely to experience adverse outcomes during follow-up.

Virtual, Augmented, and Alternate Reality in Medical Education: Socially Distanced but Fully Immersed.

BACKGROUND: Advancements in technology continue to transform the landscape of medical education. The need for technology-enhanced distance learning has been further accelerated by the coronavirus disease (COVID-19) pandemic. The relatively recent emergence of virtual reality (VR), augmented reality (AR), and alternate reality has expanded the possible applications of simulation-based education (SBE) outside of the traditional simulation laboratory, making SBE accessible asynchronously and in geographically diverse locations. OBJECTIVE: In this review, we will explore the evidence base for use of emerging technologies in SBE as well as the strengths and limitations of each modality in a variety of settings. METHODS: PubMed was searched for peer-reviewed articles published between 1995 and 2021 that focused on VR in medical education. The search terms included medical education, VR, simulation, AR, and alternate reality. We also searched reference lists from selected articles to identify additional relevant studies. RESULTS: VR simulations have been used successfully in resuscitation, communication, and bronchoscopy training. In contrast, AR has demonstrated utility in teaching anatomical correlates with the use of diagnostic imaging, such as point-of-care ultrasound. Alternate reality has been used as a tool for developing clinical reasoning skills, longitudinal patient panel management, and crisis resource management via multiplayer platforms. CONCLUSION: Although each of these modalities has a variety of educational applications in health profession education, there are benefits and limitations to each that are important to recognize prior to the design and implementation of educational content, including differences in equipment requirements, cost, and scalability.

Peer Discussion Decreases Practice Intensity and Increases Certainty in Clinical Decision-Making Among Internal Medicine Residents.

BACKGROUND: Team-based decision-making has been shown to reduce diagnostic error, increase clinical certainty, and decrease adverse events. OBJECTIVE: This study aimed to assess the effect of peer discussion on resident practice intensity (PI) and clinical certainty (CC). METHODS: A vignette-based instrument was adapted to measure PI, defined as the likelihood of ordering additional diagnostic tests, consultations or empiric treatment, and CC. Internal medicine residents at 7 programs in the Philadelphia area from April 2018 to June 2019 were eligible for inclusion in the study. Participants formed groups and completed each item of the instrument individually and as a group with time for peer discussion in between individual and group responses. Predicted group PI and CC scores were compared with measured group PI and CC scores, respectively, using paired t testing. RESULTS: Sixty-nine groups participated in the study (response rate 34%, average group size 2.88). The measured group PI score (2.29, SD = 0.23) was significantly lower than the predicted group PI score (2.33, SD = 0.22) with a mean difference of 0.04 (SD = 0.10; 95% CI 0.02-0.07; P = .0002). The measured group CC score (0.493, SD = 0.164) was significantly higher than the predicted group CC score (0.475, SD = 0.136) with a mean difference of 0.018 (SD = 0.073; 95% CI 0.0006-0.0356; P = .022). CONCLUSIONS: In this multicenter study of resident PI, peer discussion reduced PI and increased CC more than would be expected from averaging group members' individual scores.

Development and Implementation of a Novel Learner-driven Precepting Model for Pulmonary Fellowship.

Background: Ambulatory education is currently underemphasized in pulmonary and critical care medicine (PCCM) fellowship training. Existing clinic precepting models, originally developed for students and residents, do not meet the unique needs of fellow-level trainees. Objective: We aimed to develop and implement a novel fellow-led precepting model to improve ambulatory education for PCCM trainees. Methods: We performed a mixed-methods needs assessment, including surveys, focus groups, and direct observations, to explore perceptions of ambulatory training, define current precepting practices, and identify target areas for improvement. On the basis of these findings, we developed, implemented, and evaluated a novel model for PCCM outpatient precepting. Results: A targeted needs assessment identified that current precepting practices did not meet fellows' needs for graduated autonomy, development of assessment and management skills, and self-directed learning. We developed and implemented a novel, learner-driven precepting model (Set the Stage, Tell the Story, Educational Goals, Preliminary Plan, Uncertainties, Plan Recap [STEP-UP]), designed to prioritize clinical reasoning skills and self-identification of learning goals. Implementation of the STEP-UP model improved perceptions of overall outpatient training and precepting. However, we faced several barriers to uptake of the new model, including increased cognitive burden of integrating a new process. Conclusion: A robust assessment of ambulatory education at a single PCCM fellowship program identified a need to align precepting processes with the unique goals of advanced trainees. We developed a learner-driven precepting model focused on development of clinical reasoning skills and self-directed educational objectives. Additional study is warranted to refine, adapt, and test the model in different setting.

Assessment of Gender-based Qualitative Differences within Trainee Evaluations of Faculty.

Rationale: Recent studies of trainee evaluations of medical faculty have demonstrated subtle gender-based word choice differences. However, it is not known if this manifests in major contextual differences in written comments.Objectives: To characterize qualitative differences in narrative evaluations of female and male pulmonary and critical care medicine faculty.Methods: We performed a single-center, retrospective cohort analysis of trainee narrative evaluations of pulmonary and critical care medicine faculty at the Hospital of the University of Pennsylvania, written from 2015 to 2016. Directive and summative content analysis was performed by four raters. Major contextual themes were identified using constant comparative techniques. Thematic differences based on faculty gender were identified, with statistical significance determined using χ2 analysis (P < 0.05). Effect sizes were calculated using Cramér's V.Results: A total of 1,216 total narrative evaluations were analyzed, representing 62 faculty members (17 women and 45 men), with analysis continued for the full cohort beyond achieving saturation of themes. Five overarching themes emerged: teaching skills, clinical skills, supervision, interpersonal and communication skills, and leadership skills. Within subthemes, we found no significant gender differences in reference to general teaching skills, learning environment, enthusiasm for teaching, or interpersonal concern. We identified subtheme differences between male and female faculty evaluations in regard to mentions of learner autonomy, clinical learning environment, humor, and motivating the trainee for patient care. Although the mention of constructive criticisms did not differ between genders, constructive criticisms of female faculty were more likely to mention demeanor (P = 0.06).Conclusions: Our data suggest minimal thematic differences in trainee narrative evaluations of male and female pulmonary and critical care medicine faculty. However, we noted several subtheme contextual differences between male and female faculty, which warrants further investigation.