Developing and authenticating an electronic health record-based report card for assessing residents' clinical performance.

Purpose: The electronic health record (EHR) is frequently identified as a source of assessment data regarding residents' clinical performance. To better understand how to harness EHR data for education purposes, the authors developed and authenticated a prototype resident report card. This report card used EHR data exclusively and was authenticated with various stakeholders to understand individuals' reactions to and interpretations of EHR data when presented in this way. Methods: Using principles derived from participatory action research and participatory evaluation, this study brought together residents, faculty, a program director, and medical education researchers (n = 19) to develop and authenticate a prototype report card for residents. From February to September 2019, participants were invited to take part in a semistructured interview that explored their reactions to the prototype and provided insights about how they interpreted the EHR data. Results: Our results highlighted three themes: data representation, data value, and data literacy. Participants varied in terms of the best way to present the various EHR metrics and felt pertinent contextual information should be included. All participants agreed that the EHR data presented were valuable, but most had concerns about using it for assessment. Finally, participants had difficulties interpreting the data, suggesting that these data could be presented more intuitively and that residents and faculty may require additional training to fully appreciate these EHR data. Conclusions: This work demonstrated how EHR data could be used to assess residents' clinical performance, but it also identified areas that warrant further consideration, especially pertaining to data representation and subsequent interpretation. Providing residents and faculty with EHR data in a resident report card was viewed as most valuable when used to guide feedback and coaching conversations.

Facilitating Residents' Understanding of Electronic Health Record Report Card Data Using Faculty Feedback and Coaching.

PURPOSE: Feedback continues to present a challenge for competency-based medical education. Clear, consistent, and credible feedback is vital to supporting one's ongoing development, yet it can be difficult to gather clinical performance data about residents. This study sought to determine whether providing residents with electronic health record (EHR)-based report cards, as well as an opportunity to discuss these data with faculty trained using the R2C2 model, can help residents understand and interpret their clinical performance metrics. METHOD: Using action research methodology, the author team collected EHR data from July 2017 to February 2020, for all residents (n = 21) in one 5-year Emergency Medicine program and created personalized report cards for each resident. During October 6-17, 2020, 8 out of 17 eligible residents agreed to have their feedback conversations recorded and participate in a subsequent interview with a nonphysician member of the research team. Data were analyzed using thematic analysis, and the authors used inductive analysis to identify themes in the data. RESULTS: In analyzing both the feedback conversations as well as the individual interviews with faculty and residents, the authors identified 2 main themes: (1) Reactions and responses to receiving personalized EHR data and (2) The value of EHR data for assessment and feedback purposes. All participants believed that EHR data metrics are useful for prompting self-reflection, and many pointed to their utility in providing suggestions for actionable changes in their clinical practice. For faculty, having a tool through which underperforming residents can be shown "objective" data about their clinical performance helps underscore the need for improvement, particularly when residents are resistant. CONCLUSIONS: The EHR is a valuable source of educational data, and this study demonstrates one of the many thoughtful ways it can be used for assessment and feedback purposes.

"EMERGing" Electronic Health Record Data Metrics: Insights and Implications for Assessing Residents' Clinical Performance in Emergency Medicine.

OBJECTIVES: Competency-based medical education requires that residents are provided with frequent opportunities to demonstrate competence as well as receive effective feedback about their clinical performance. To meet this goal, we investigated how data collected by the electronic health record (EHR) might be used to assess emergency medicine (EM) residents' independent and interdependent clinical performance and how such information could be represented in an EM resident report card. METHODS: Following constructivist grounded theory methodology, individual semistructured interviews were conducted in 2017 with 10 EM faculty and 11 EM residents across all 5 postgraduate years. In addition to open-ended questions, participants were presented with an emerging list of EM practice metrics and asked to comment on how valuable each would be in assessing resident performance. Additionally, we asked participants the extent to which each metric captured independent or interdependent performance. Data collection and analysis were iterative; analysis employed constant comparative inductive methods. RESULTS: Participants refined and eliminated metrics as well as added new metrics specific to the assessment of EM residents (e.g., time between signup and first orders). These clinical practice metrics based on data from our EHR database were organized along a spectrum of independent/interdependent performance. We conclude with discussions about the relationship among these metrics, issues in interpretation, and implications of using EHR for assessment purposes. CONCLUSIONS: Our findings document a systematic approach for developing EM resident assessments, based on EHR data, which incorporate the perspectives of both clinical faculty and residents. Our work has important implications for capturing residents' contributions to clinical performances and distinguishing between independent and interdependent metrics in collaborative workplace-based settings.

Elucidating system-level interdependence in electronic health record data: What are the ramifications for trainee assessment?

CONTEXT: The electronic health record (EHR) has been identified as a potential site for gathering data about trainees' clinical performance, but these data are not collected or organised for this purpose. Therefore, a careful and rigorous approach is required to explore how EHR data could be meaningfully used for assessment purposes. The purpose of this study was to identify EHR performance metrics that represent both the independent and interdependent clinical performance of emergency medicine (EM) trainees and explore how they might be meaningfully used for assessment and feedback. METHODS: Using constructivist grounded theory, we conducted 21 semi-structured interviews with EM faculty members and residents. Participants were asked to identify the clinical actions of trainees that would be valuable for assessment and feedback and describe how those activities are represented in the EHR. Data collection and analysis, which consisted of three stages of coding, occurred iteratively. RESULTS: When faculty members and trainees in EM were asked to reflect on the usefulness of using EHR performance metrics for resident assessment and feedback they expressed both widespread support for the idea in principle and hesitation that aspects of clinical performance captured in the data would not be representative of residents' individual performance, but would rather reflect their interdependence with other team members and the systems in which they work. We highlight three categorisations of system-level interdependence - medical directives, technological systems and organisational systems - identified by our participants, and discuss strategies participants employed to navigate these forms of interdependence within the health care system. CONCLUSIONS: System-level interdependence shapes physicians' performances, and yet, this impact is rarely corrected for or noted within clinical performance data. Educators have a responsibility to recognise system-level interdependence when teaching and consider system-level interdependence when assessing the performance of trainees in order to most effectively and fairly utilise the EHR as a source of assessment data.

How do emergency departments and emergency leaders catalyze positive change through quality improvement collaborations?

OBJECTIVES: Quality Improvement and Patient Safety (QIPS) plays an important role in addressing shortcomings in optimal healthcare delivery. However, there is little published guidance available for emergency department (ED) teams with respect to developing their own QIPS programs. We sought to create recommendations for established and aspiring ED leaders to use as a pathway to better patient care through programmatic QIPS activities, starting internally and working towards interdepartmental collaboration. METHODS: An expert panel comprised of ten ED clinicians with QIPS and leadership expertise was established. A scoping review was conducted to identify published literature on establishing QIPS programs and frameworks in healthcare. Stakeholder consultations were conducted among Canadian healthcare leaders, and recommendations were drafted by the expert panel based on all the accumulated information. These were reviewed and refined at the 2018 CAEP Academic Symposium in Calgary using in-person and technologically-supported feedback. RESULTS: Recommendations include: creating a sense of urgency for improvement; engaging relevant stakeholders and leaders; creating a formal local QIPS Committee; securing funding and resources; obtaining local data to guide the work; supporting QIPS training for team members; encouraging interprofessional, cross-departmental, and patient collaborations; using an established QIPS framework to guide the work; developing reward mechanisms and incentive structures; and considering to start small by focusing on a project rather than a program. CONCLUSION: A list of 10 recommendations is presented as guiding principles for the establishment and sustainable deployment of QIPS activities in EDs throughout Canada and abroad. ED leaders are encouraged to implement our recommendations in an effort to improve patient care.

OBJECTIF: L'amélioration de la qualité et la sécurité des patients (AQSP) joue un rôle important dans la correction des lacunes observées dans la prestation optimale de soins. Toutefois, les équipes de soins au service des urgences (SU) disposent de peu de documentation sur la conception de leurs propres programmes d'AQSP. L'étude avait donc pour objectif l'élaboration de recommandations conçues à l'intention des chefs de file, nouveaux ou confirmés, au SU, et présentées comme une voie à emprunter pour améliorer les soins aux patients, par l'application d'activités programmatiques d'AQSP, tout d'abord au sein du service, puis entre services, grâce à la collaboration. MÉTHODE: Un groupe d'experts composé de 10 cliniciens en médecine d'urgence, ayant des compétences particulières en AQSP et en pouvoir d'influence, a été mis sur pied. Un examen de cadrage a été entrepris à la recherche de publications sur l'établissement de programmes d'AQSP et de cadres de travail s'y rapportant, en soins de santé. Des consultations ont été menées avec les parties intéressées parmi les chefs de file en soins de santé au Canada, et le groupe d'experts a rédigé une version préliminaire de recommandations fondées sur l'ensemble de l'information recueillie. Celles-ci ont été examinées et améliorées durant le Symposium sur les affaires universitaires 2018 de l'ACMU, à Calgary, à la suite de rétroactions communiquées en personne ou par voie électronique. RÉSULTATS: Les recommandations portaient sur : l'éveil d'un sentiment d'urgence à l'égard de l'amélioration; la mobilisation d'intervenants et de chefs de file compétents; la mise sur pied d'un comité local structuré d'AQSP; l'obtention de financement et de ressources; la disponibilité de données locales pour orienter le travail; le soutien de la formation des membres d'équipe en AQSP; la promotion de la collaboration entre professions, entre services et avec les patients; l'utilisation d'un cadre de travail d'AQSP déjà établi afin d'orienter le travail; l'élaboration d'un système de récompenses et de structures incitatives; la possibilité d'entreprendre, au début, des initiatives à petite échelle, soit des projets plutôt que des programmes. CONCLUSION: Les dix recommandations ont été présentées à titre de principes directeurs en vue de l'élaboration d'activités d'AQSP et de leur mise en œuvre durable dans les SU, au Canada et ailleurs dans le monde. Les chefs de file dans les SU sont invités à appliquer ces recommandations dans le but d'améliorer les soins aux patients.

Effects of nebulized diethylenetetraamine-NONOate in a mouse model of acute Pseudomonas aeruginosa pneumonia.

OBJECTIVE: Endogenous and exogenous nitric oxide (NO) may have important antibacterial effects in patients with pneumonia. NO administration has been limited to the continuous inhalation of gas-phase NO (ie, inhaled NO [iNO]). Intermittent nebulization of NONOates, novel NO donors, may permit the continuous intrapulmonary delivery of NO. Thus, we assessed the effects of nebulized diethylenetetraamine-NONOate (DETA-NO) in a model of acute Pseudomonas aeruginosa pneumonia. DESIGN: Randomized, controlled study. SUBJECTS: Male C57Bl/6 mice. INTERVENTIONS: Pneumonia was induced by intratracheal instillation of P aeruginosa (3 x 10(7) CFU in 50 microL). Pneumonia and sham mice were randomized to receive no treatment, nebulized DETA-NO (12.5 or 125 micromol) at 4 h and 12 h, or continuous iNO for 24 h (10 or 40 ppm) until they were killed at 24 h. MAIN RESULTS: The nebulization of DETA-NO was associated with a marked increase in mean (+/- SEM) exhaled NO levels (after nebulization, 484 +/- 34 parts per billion [ppb]; baseline, 13.4 +/- 0.4 ppb; p < 0.01) and plasma levels of nitrites/nitrates (after nebulization, 73 +/- 28 microM; at baseline, 14 +/- 3 microM; p < 0.05). Nebulized DETA-NO decreased the pulmonary bacterial load in mice with pneumonia by 65 +/- 19% (p < 0.05 vs untreated mice) but had no effect on pulmonary leukocyte infiltration. Although the growth of P aeruginosa colonies in vitro was impaired on exposure to DETA-NO, growth was similarly impaired by exposure to DETA nucleophile/backbone alone. CONCLUSIONS: The nebulization of DETA-NO provides a method for the prolonged intrapulmonary delivery of NO. The antibacterial effect of DETA-NO in vivo and in vitro is due, in large part, to the DETA nucleophile moiety and is independent of NO, suggesting a limited therapeutic role for exogenous NO in pneumonia.