Perceptions of Justice in Clinical Learning Environments: Development and Validation of an Organizational Justice Measure for Medical Trainees.

PURPOSE: This study aimed to develop an instrument to measure medical trainees' perceptions of justice in clinical learning environments. METHOD: Between 2019 and 2023, the authors conducted a multiyear, multi-institutional, multiphase study to develop a 16-item justice measure with 4 dimensions: interpersonal, informational, procedural, and distributive. The authors gathered validity evidence based on test content, internal structure, and relationships with other variables across 3 phases. Phase 1 involved drafting items and gathering evidence that items measured intended dimensions. Phase 2 involved analyzing relevance of items for target groups, examining interitem correlations and factor loadings in a preliminary analysis, and obtaining reliability estimates. Phase 3 involved a confirmatory factor analysis and collecting convergent and discriminant validity evidence. RESULTS: In phase 1, 63 of 91 draft items were retained following a content validation exercise gauging how well items measured targeted dimensions (mean [SD] item ratings within dimensions, 4.16 [0.36] to 4.39 [0.34]) on a 5-point Likert scale (with 1 indicating not at all well and 5 indicating extremely well). In phase 2, 30 items were removed due to low factor loadings (i.e., < 0.40), and 4 items per dimension were selected (factor loadings, 0.42-0.89). In phase 3, a confirmatory factor analysis supported the 4-dimension model (χ2 = 610.14, P < .001; comparative fit index = 0.90, Tucker-Lewis Index = 0.87, root mean squared error of approximation = 0.11, standardized root mean squared residual = 0.06), with convergent and discriminant validity evidence showing hypothesized positive correlations with a justice measure (r = 0.93, P < .001), trait positive affect (r = 0.46, P < .001), and emotional stability (r = 0.33, P < .001) and negative correlations with trait negative affect (r = -0.39, P < .001). CONCLUSIONS: Results indicate the measure's potential utility in understanding justice perceptions and designing targeted interventions.

Mobile technologies to support workplace-based assessment for entrustment decisions: Guidelines for programs and educators: AMEE Guide No. 154.

With the rise of competency-based medical education and workplace-based assessment (WBA) since the turn of the century, much has been written about methods of assessment. Direct observation and other sources of information have become standard in many clinical programs. Entrustable professional activities (EPAs) have also become a central focus of assessment in the clinical workplace. Paper and pencil (one of the earliest mobile technologies!) to document observations have become almost obsolete with the advent of digital technology. Typically, clinical supervisors are asked to document assessment ratings using forms on computers. However, accessing these forms can be cumbersome and is not easily integrated into existing clinical workflows. With a call for more frequent documentation, this practice is hardly sustainable, and mobile technology is quickly becoming indispensable. Documentation of learner performance at the point of care merges WBA with patient care and WBA increasingly uses smartphone applications for this purpose.This AMEE Guide was developed to support institutions and programs who wish to use mobile technology to implement EPA-based assessment and, more generally, any type of workplace-based assessment. It covers backgrounds of WBA, EPAs and entrustment decision-making, provides guidance for choosing or developing mobile technology, discusses challenges and describes best practices.

Evidence for validity of the epidemic-pandemic impacts inventory (brief healthcare module): Internal structure and association with other variables.

The COVID-19 pandemic has subjected healthcare workers to enormous stress. Measuring the impact of this public health emergency is essential to developing strategies that can effectively promote resilience and wellness. The Epidemic-Pandemic Impacts Inventory Supplemental Healthcare Module-Brief Version (EPII-SHMb) was developed to measure impacts among occupational cohorts serving on the front lines of healthcare. While this instrument has been utilized in COVID-19 related studies, little is known about its psychometric properties. This study collects evidence for validity of the EPII-SHMb by evaluating its internal structure and how its scores associate with other variables. Physicians and nursing staff across a large New York health system were cross-sectionally surveyed using an online questionnaire between June and November 2020. Exploratory factor analysis resulted in a 3-factor solution, identifying factors Lack of Workplace Safety (7 items), Death/Dying of Patients (3 items), and Lack of Outside Support (2 items). Internal consistency was high overall and within physician/nursing and gender subgroups (Cronbach's alpha: 0.70 - 0.81). Median scores on Death/Dying of Patients were higher among those who directly cared for COVID-19 patients or worked in COVID-19 hospital units. These results are promising. Additional studies evaluating other dimensions of validity are necessary.

Physician challenges and supports during the first wave of the COVID-19 pandemic: A mixed methods study.

Introduction: During the COVID-19 pandemic, physicians encountered significant COVID-19-related negative experiences and psychological distress in both their personal and professional lives. To understand the factors that negatively impact physician well-being, a number of studies have pointed to multiple work system factors such as excessive workload and workflow interruptions. In addition, studies have shown that positive interpersonal relationships that provide social support may also serve as a buffering role against psychological distress. The aim of our study explores the challenges and sources of support for physicians relative to mental health symptoms. Methods: In this study, We used a cross-sectional study design with a convergent parallel mixed method approach combining both qualitative and quantitative data collected in parallel from a self-report questionnaire immediately following the first wave of COVID-19. The aim of our study explores the challenges and sources of support for physicians relative to mental health symptoms. Results: Of the 457 physicians in the study, the most frequently potential negative occupational experiences were, "Being at risk of contracting COVID-19 from patients/co-workers" (90.5%) and "Contact with distressed family members who cannot be with a loved one" (69.5%). We identified five common themes for main sources of social support (e.g. emotional support from family/friends) and six themes for challenges (e.g., work-related demands exacerbated by the pandemic). Discussion: Our study highlights COVID-19 and other pandemic-related challenges that negatively impacted the mental health of physicians. Interventions that provide targeted organizational supports (e.g. sufficient PPE and child support), as well as specific sources of support (e.g. family and emotional), can attenuate those challenges and stressors experienced during a pandemic.

Can a Situational Judgment Test Identify Trainees at Risk of Professionalism Issues? A Multi-Institutional, Prospective Cohort Study.

PURPOSE: To determine whether overall situational judgment test (SJT) scores are associated with programs' clinical competency committee (CCC) ratings of trainee professionalism, any concerning behavior, and concerning behavior requiring active remediation at 2 time periods. METHOD: In fall 2019, trainees from 17 U.S. programs (16 residency, 1 fellowship) took an online 15-scenario SJT developed to measure 7 dimensions of professionalism. CCC midyear and year-end (6 months and 1 year following SJT completion, respectively) professionalism scores and concern ratings were gathered for academic year 2019-2020. Analyses were conducted to determine whether overall SJT scores related to overall professionalism ratings, trainees displaying any concerns, and trainees requiring active remediation at both time periods. RESULTS: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings ( r = .21 and .14, P < .001 and = .03, respectively). Holding gender and race/ethnicity constant, a 1 standard deviation (SD) increase in overall SJT score was associated with a .20 SD increase in overall professionalism ratings at midyear ( P = .005) and a .22 SD increase at year-end ( P = .001). Holding gender and race/ethnicity constant, a 1 SD increase in overall SJT score decreased the odds of a trainee displaying any concerns by 37% (odds ratio [OR] 95% confidence interval [CI]: [.44, .87], P = .006) at midyear and 34% (OR 95% CI: [.46, .95], P = .025) at year-end and decreased the odds of a trainee requiring active remediation by 51% (OR 95% CI: [.25, .90], P = .02) at midyear. CONCLUSIONS: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings and were associated with whether trainees exhibited any concerning behavior at midyear and year-end and whether trainees needed active remediation at midyear. Future research should investigate whether other potential professionalism measures are associated with concerning trainee behavior.

Instructional Design Strategies for Teaching the Mental Status Examination and Psychiatric Interview: a Scoping Review.

OBJECTIVE: The psychiatric mental status examination is a fundamental aspect of the psychiatric clinical interview. However, despite its importance, little emphasis has been given to evidence-based instructional design. Therefore, this review summarizes the literature from an instructional design perspective with the aim of uncovering design strategies that have been used for teaching the psychiatric interview and mental status examination to health professionals. METHODS: The authors conducted a scoping review. Multiple databases, reference lists, and the gray literature were searched for relevant publications across educational levels and professions. A cognitive task analysis and an instructional design framework was used to summarize and chart the findings. RESULTS: A total of 61 articles from 17 countries in six disciplines and three educational levels were identified for data extraction and analysis. Most studies were from the USA, presented as educational case reports, and carried out in undergraduate education in the field of psychiatry. Few articles described the instructional rationale for their curriculum. None of the studies compared the effectiveness of different instructional design components. Reported learning activities for each task domain (knowledge, skills, and attitudes) and for each step of an instructional design process were charted. Most articles reported the use of introductory seminars or lectures in combination with digital learning material (videos and virtual patients in more recent publications) and role-play exercises. CONCLUSIONS: Educators in psychiatry should consider all task domains of the psychiatric interview and mental status examination. Currently, there is a lack of empirical research on expertise acquisition and use of instructional design frameworks in this context.

The Effect of the COVID-19 Pandemic on Pediatric Physician Wellness: A Cross-Sectional Study.

The COVID-19 pandemic has provided challenges to all healthcare workers. While the brunt of treating COVID-19 patients fell upon adult providers, pediatricians also experienced significant stressors and disruptions. Academic pediatricians and trainees (fellows and residents) were redeployed to manage adult patients in hospitalist and intensive care settings and/or had major changes to their clinical schedules. In this study, we aimed to describe levels of self-reported depression, anxiety, and burnout in pediatric physicians following the initial wave of the pandemic at the largest integrated health system in New York State. A cross-sectional study was conducted among pediatric physicians who cared for patients during the COVID-19 pandemic within the Northwell Health System as part of the Northwell Wellbeing Registry, a longitudinal registry assessing the psychological impact of COVID-19 on healthcare providers. A total of 99 pediatric physician respondents were included in this study; 72% of whom were attendings, 28% of whom were trainees. Compared to attendings, trainees reported significantly higher proportions of burnout-emotional exhaustion (p = 0.0007) and burnout-depersonalization (p = 0.0011) on the Abbreviated Maslach Burnout Inventory. There was not a similar trend in probable depression or probable anxiety using the Patient Health Questionnaire. In a multivariable logistic regression model, being a trainee was significantly associated with increased odds of burnout-emotional exhaustion (OR 5.94, 95% Confidence Interval: 1.85-19.02). These findings suggest that fellows and residents were a vulnerable population during the COVID-19 pandemic. Training programs should pay special attention to their trainees during times of crisis, and future studies can help to identify protective factors to reduce the risk of burnout during these times.

The Impact of Physicians' COVID-19 Pandemic Occupational Experiences on Mental Health.

OBJECTIVE: To examine the association between a number of negative COVID-19 occupational experiences and probable anxiety, depression, and PTSD among physicians. METHODS: Cross-sectional examination of longitudinal registry data consisting of physician personal and occupational well-being. Multivariable logistic regressions were performed to determine the association between negative COVID-19 experiences and outcomes. RESULTS: Of the 620 eligible physicians, approximately half were female (49%), and 71% white with a mean age of 46.51 (SD = 13.28). A one-point increase in negative experience score was associated with a 23% increase in probable anxiety (OR = 1.23, 95% CI: 1.14-1.34), a 23% increase in probable depression (OR = 1.23, 95% CI: 1.13-1.33), and a 41% increase in probable PTSD (OR = 1.41, 95% CI: 1.30-1.52). CONCLUSIONS: Negative pandemic experiences were strongly associated with adverse mental health outcomes while greater resilience was protective.

Evaluating burnout during the COVID-19 pandemic among physicians in a large health system in New York.

PURPOSE: The COVID-19 pandemic has generated significant psychological distress among health care workers worldwide. New York State, particularly New York City and surrounding counties, were especially affected, and experienced over 430,000 COVID-19 cases and 25,000 deaths by mid-August 2020. We hypothesized that physicians and trainees (residents/fellows) who were redeployed outside of their specialty to treat COVID-19 inpatients would have higher burnout. METHODS: We conducted a cross-sectional survey to assess burnout among attending and trainee physicians who provided patient care during the COVID-19 pandemic between March-May 2020 across a diverse health care system in New York. Separate multivariable logistic regressions were performed to determine the association between redeployment and measures of burnout: Emotional Exhaustion (EE) and Depersonalization. Burnout measures were also compared by physician vs trainee status. The differential association between redeployment and outcomes with respect to trainee status was also evaluated. RESULTS: Redeployment was significantly associated with increased odds of EE {OR =1.53, 95% CI: 1.01-2.31} after adjusting for gender and Epidemic-Pandemic Impacts Inventory (EPII) score. Similarly, being a trainee, especially a junior level trainee, was associated with increased odds of EE {OR = 1.59, 95% CI: 1.01-2.51} after adjusting for gender and EPII scores. However, neither redeployment nor trainee status were significantly associated with Depersonalization. Interactions between redeployment and trainee status were not significant for any of the outcomes (p>.05). CONCLUSION: Physicians who were redeployed to treat COVID-19 patients had higher reported measures of EE. Trainees, irrespective of redeployment status, had higher EE as compared with attendings. Additional research is needed to understand the long-term impact of redeployment on burnout among redeployed physicians. Programs to identify and address potential burnout among physicians, particularly trainees, during pandemics may be beneficial.

Supervision and Care Quality as Perceived by Redeployed Attendings, Fellows, and Residents During a COVID-19 Surge: Lessons for the Future.

PURPOSE: To better prepare for potential future large-scale redeployments, this study examines quality of supervision and care as perceived by redeployed residents, fellows, and attendings during a COVID-19 surge. METHOD: During April and May 2020, attendings, fellows, and residents redeployed at 2 teaching hospitals were invited to participate in a survey, which included questions on respondents' prior experience; redeployed role; amount of supervision needed and received; and perceptions of quality of supervision, patient care, and interprofessional collaboration. Frequencies, means, and P values were calculated to compare perceptions by experience and trainee status. Narrative responses to 2 open-ended questions were independently coded; themes were constructed. RESULTS: Overall, 152 of 297 (51.2%) individuals responded, including 64 of 142 attendings (45.1%), 40 of 79 fellows (50.6%), and 48 of 76 residents (63.2%). Fellows and attendings, regardless of prior experience, perceived supervision as adequate. In contrast, experienced residents reported receiving more supervision than needed, while inexperienced residents reported receiving less supervision than needed and rated overall supervision as poor. Attendings, fellows, and experienced residents rated the overall quality of care as acceptable to good, whereas inexperienced residents perceived overall quality of care as worse to much worse, particularly when compared with baseline. CONCLUSIONS: Narrative themes indicated that the quality of supervision and care was buffered by strong camaraderie, a culture of informal consultation, team composition (mixing experienced with inexperienced), and clinical decision aids. The markedly negative view of inexperienced residents suggests a higher risk for disillusionment, perhaps even moral injury, during future redeployments. Implications for planning are explored.

Burnout and the Quantified Workplace: Tensions around Personal Sensing Interventions for Stress in Resident Physicians.

Recent research has explored computational tools to manage workplace stress via personal sensing, a measurement paradigm in which behavioral data streams are collected from technologies including smartphones, wearables, and personal computers. As these tools develop, they invite inquiry into how they can be appropriately implemented towards improving workers' well-being. In this study, we explored this proposition through formative interviews followed by a design provocation centered around measuring burnout in a U.S. resident physician program. Residents and their supervising attending physicians were presented with medium-fidelity mockups of a dashboard providing behavioral data on residents' sleep, activity and time working; self-reported data on residents' levels of burnout; and a free text box where residents could further contextualize their well-being. Our findings uncover tensions around how best to measure workplace well-being, who within a workplace is accountable for worker stress, and how the introduction of such tools remakes the boundaries of appropriate information flows between worker and workplace. We conclude by charting future work confronting these tensions, to ensure personal sensing is leveraged to truly improve worker well-being.

Supervision, Interprofessional Collaboration, and Patient Safety in Intensive Care Units during the COVID-19 Pandemic.

Background: To meet coronavirus disease (COVID-19) demands in the spring of 2020, many intensive care (IC) units (ICUs) required help of redeployed personnel working outside their regular scope of practice, causing an expansion and change of staffing ratios. Objective: How did this composite alternative ICU workforce experience supervision, interprofessional collaboration, and quality and safety of care under the unprecedented clinical circumstances at the height of the first pandemic wave as lived experiences uniquely captured during the first peak of the pandemic? Methods: An international, cross-sectional survey was conducted among physicians, nurses, and allied personnel deployed or redeployed to ICUs in Utrecht, New York, and Dublin from April to May of 2020. Data were analyzed separately for the three sites. Quantitative data were treated for descriptive statistics; qualitative data were analyzed thematically and combined for general interpretations. Results: On the basis of 234, 83, and 34 responses (response rates of 68%, 48%, and 41% in Utrecht, New York, and Dublin, respectively), we found that the amount of supervision and the quality and safety of care were perceived as being lower than usual but still acceptable. The working atmosphere was overwhelmingly felt to be collaborative and supportive. Where IC-certified nurse-to-patient ratios had decreased most (Utrecht), nurses voiced criticism about supervision and quality of care. Continuity within the work environment, team composition, and informal ("curbside") consultations were critical mediators of success. Conclusion: In the exceptional circumstances encountered during the COVID-19 pandemic, many ICUs were managed by a composite workforce of IC-certified and redeployed personnel. Although supervision is critical for safe care, supervisory roles were not clearly related to the amount of prior ICU experience. Vital for satisfaction with the quality of care was the span of control for those who assumed supervisory roles (i.e., the ratio of certified to noncertified personnel). Stable teams that matched less experienced personnel with more experienced personnel; a strong, interprofessional, collaborative atmosphere; a robust culture of informal consultation; and judicious, more flexible use of rules and regulations proved to be essential.

Exploring the relationship between emotion and cognitive load types during patient handovers.

Cognitive Load Theory has emerged as an important approach to improving instruction in the health professions workplace, including patient handovers. At the same time, there is growing recognition that emotion influences learning through numerous cognitive processes including motivation, attention, working memory, and long-term memory. This study explores how emotion influences the cognitive load experienced by trainees performing patient handovers. From January to March 2019, 693 (38.7%) of 1807 residents and fellows from a 24-hospital health system in New York city completed a survey after performing a handover. Participants rated their emotional state and cognitive load. The survey included questions about features of the learner, task, and instructional environment. The authors used factor analysis to identify the core dimensions of emotion. Regression analyses explored the relationship between the emotion factors and cognitive load types. Two emotion dimensions were identified representing invigoration and tranquility. In regression analyses, higher levels of invigoration, tranquility, and their interaction were independently associated with lower intrinsic load and extraneous load. The interaction of invigoration and tranquility predicted lower germane load. The addition of the emotion variables to multivariate models including other predictors of cognitive load types significantly increased the amount of variance explained. The study provides a model for measuring emotions in workplace learning. Because emotion appears to have a significant influence on cognitive load types, instructional designers should consider strategies that help trainees regulate emotion in order to reduce cognitive load and improve learning and performance.

Competency-Based Assessment in Psychiatric Education: A Systems Approach.

Medical education programs are failing to meet the health needs of patients and communities. Misalignments exist on multiple levels, including content (what trainees learn), pedagogy (how trainees learn), and culture (why trainees learn). To address these challenges effectively, competency-based assessment (CBA) for psychiatric medical education must simultaneously produce life-long learners who can self-regulate their own growth and trustworthy processes that determine and accelerate readiness for independent practice. The key to effectively doing so is situating assessment within a carefully designed system with several, critical, interacting components: workplace-based assessment, ongoing faculty development, learning analytics, longitudinal coaching, and fit-for-purpose clinical competency committees.

Advancing Workplace-Based Assessment in Psychiatric Education: Key Design and Implementation Issues.

With the adoption of competency-based medical education, assessment has shifted from traditional classroom domains of knows and knows how to the workplace domain of doing. This workplace-based assessment has 2 purposes; assessment of learning (summative feedback) and the assessment for learning (formative feedback). What the trainee does becomes the basis for identifying growth edges and determining readiness for advancement and ultimately independent practice. High-quality workplace-based assessment programs require thoughtful choices about the framework of assessment, the tools themselves, the platforms used, and the contexts in which the assessments take place, with an emphasis on direct observation.

Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?

The COVID-19 pandemic has disrupted many societal institutions, including health care and education. Although the pandemic's impact was initially assumed to be temporary, there is growing conviction that medical education might change more permanently. The International Competency-based Medical Education (ICBME) collaborators, scholars devoted to improving physician training, deliberated how the pandemic raises questions about medical competence. We formulated 12 broad-reaching issues for discussion, grouped into micro-, meso-, and macro-level questions. At the individual micro level, we ask questions about adaptability, coping with uncertainty, and the value and limitations of clinical courage. At the institutional meso level, we question whether curricula could include more than core entrustable professional activities (EPAs) and focus on individualized, dynamic, and adaptable portfolios of EPAs that, at any moment, reflect current competence and preparedness for disasters. At the regulatory and societal macro level, should conditions for licensing be reconsidered? Should rules of liability be adapted to match the need for rapid redeployment? We do not propose a blueprint for the future of medical training but rather aim to provoke discussions needed to build a workforce that is competent to cope with future health care crises.

The medical pause: Importance, processes and training.

Research has shown that taking 'timeouts' in medical practice improves performance and patient safety. However, the benefits of taking timeouts, or pausing, are not sufficiently acknowledged in workplaces and training programmes. To promote this acknowledgement, we suggest a systematic conceptualisation of the medical pause, focusing on its importance, processes and implementation in training programmes. By employing insights from educational and cognitive psychology, we first identified pausing as an important skill to interrupt negative momentum and bolster learning. Subsequently, we categorised constituent cognitive processes for pausing skills into two phases: the decision-making phase (determining when and how to take pauses) and the executive phase (applying relaxation or reflection during pauses). We present a model that describes how relaxation and reflection during pauses can optimise cognitive load in performance. Several strategies to implement pause training in medical curricula are proposed: intertwining pause training with training of primary skills, providing second-order scaffolding through shared control and employing auxiliary tools such as computer-based simulations with a pause function.

Psychiatry Clinical Skills Evaluation: a Multisite Study of Validity.

OBJECTIVE: Since 2007, the American Board of Psychiatry and Neurology (ABPN) has required that residency programs conduct a specific clinical skills evaluation (CSE) of physician-patient interaction, psychiatric interview and mental status examination, and case presentation on a directly observed patient interview as a prerequisite for certification. The authors examined a multisite database of CSE assessments to investigate the validity of the evaluation. METHODS: The authors collected 1156 CSE assessments from 4 residency programs conducted over a 6-year period, compared scoring patterns among the programs, score improvement over 4 years of residency, time and number of CSEs required to meet ABPN requirements, and patterns of scoring for individual faculty evaluators. RESULTS: The distribution of scores within each of the 4 programs showed similar, but nonidentical patterns. The number of CSEs required to meet the ABPN standards (3.5) and the point in training at which this was completed (late PGY-2) were the same in all programs. CSE scores were highly correlated with year of training but were not correlated with performance on an unrelated cognitive examination. Individual faculty members tended to stay within a moderate range of scores over multiple residents, partially attributable to year of training. CONCLUSIONS: Taken together, these findings support the validity of the CSE as a measure of residents' clinical skills in the specified areas and demonstrate a moderate-high degree of consistency in the scoring of the CSE across these 4 programs.