Competence committees decision-making; an interplay of data, group orientation, and intangible impressions.

BACKGROUND: The implementation of competency-based medical education and utilization of competence committees (CC) represents a paradigm shift in residency education. This qualitative study aimed to explore the methods used by two operational CC and their members to make decisions about progression and competence of their residents. METHODS: An instrumental case study methodology was used to study the CC of two postgraduate training programs. Transcripts from observed CC meetings, member interviews, and guiding documents were analyzed using a constructivist grounded theory approach to reveal themes explaining the decision-making process. RESULTS: Our study found that the CC followed a process that began within a social decision schema model and evolved to a discussion that invoked social influence theory, shared mental models, and social judgment scheme to clarify the points of contention. We identified that the CC decision-making was at risk of bias, primarily influenced by the group composition, the group orientation and individual members' mindset, as well as their personal experiences with the trainees. CONCLUSIONS: Increased awareness of the sources of bias in CC functioning and familiarity with the CC role in competency-based medical education would enable committees to provide valuable feedback to all trainees regardless of their trajectory.

Neurological and Head/Eyes/Ears/Nose/Throat Manifestations of COVID-19: A Systematic Review and Meta-Analysis.

BACKGROUND/OBJECTIVE: Coronavirus disease 2019 (COVID-19) has been associated with various neurological and atypical head/eyes/ears/nose/throat (HEENT) manifestations. We sought to review the evidence for these manifestations. METHODS: In this systematic review and meta-analysis, we compiled studies published until March 31, 2021 that examined non-respiratory HEENT, central, and peripheral nervous system presentations in COVID-19 patients. We included 477 studies for qualitative synthesis and 59 studies for meta-analyses. RESULTS: Anosmia, ageusia, and conjunctivitis may precede typical upper/lower respiratory symptoms. Central nervous system (CNS) manifestations include stroke and encephalopathy, potentially with brainstem or cranial nerve involvement. MRI studies support CNS para-/postinfectious etiologies, but direct neuroinvasion seems very rare, with few cases detecting Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) in the CNS. Peripheral nervous system (PNS) manifestations include muscle damage, Guillain-Barre syndrome (GBS), and its variants. There was moderate-to-high study heterogeneity and risk of bias. In random-effects meta-analyses, anosmia/ageusia was estimated to occur in 56% of COVID-19 patients (95% CI: 0.41-0.71, I2:99.9%), more commonly than in patients without COVID-19 (OR: 14.28, 95% CI: 8.39-24.29, I2: 49.0%). Neurological symptoms were estimated to occur in 36% of hospitalized patients (95% CI: 0.31-0.42, I2: 99.8%); ischemic stroke in 3% (95% CI: 0.03-0.04, I2: 99.2%), and GBS in 0.04% (0.033%-0.047%), more commonly than in patients without COVID-19 (OR[stroke]: 2.53, 95% CI: 1.16-5.50, I2: 76.4%; OR[GBS]: 3.43,1.15-10.25, I2: 89.1%). CONCLUSIONS: Current evidence is mostly from retrospective cohorts or series, largely in hospitalized or critically ill patients, not representative of typical community-dwelling patients. There remains a paucity of systematically gathered prospective data on neurological manifestations. Nevertheless, these findings support a high index of suspicion to identify HEENT/neurological presentations in patients with known COVID-19, and to test for COVID-19 in patients with such presentations at risk of infection.

The effectiveness of a multifaceted, group-facilitated audit and feedback intervention to increase tranexamic acid use during total joint arthroplasty.

PURPOSE: Intraoperative tranexamic acid (TXA) is used to reduce blood loss and the need for transfusions following total hip arthroplasty (THA) and total knee arthroplasty (TKA). Despite evidence in literature and local practice protocols supporting TXA as a part of standard of care for joint arthroplasty, TXA administration is underutilized. We aimed to use group-facilitated audit and feedback as the foundation of a knowledge translation strategy to increase TXA use for THA and TKA procedures. METHODS: Anesthesiologists consented to receive two data reports summarizing their individual rates of TXA use and postoperative blood transfusions compared with site peers. Variables collected included patient demographics, TXA usage, and the frequency and volume of red blood cell transfusions administered in the 72-hr postoperative period. The facilitated feedback session discussed report findings and focused on factors contributing to local practice patterns and opportunities for change. RESULTS: Tranexamic acid use increased for THA procedures at the intervention site from 66.6 to 74.4% (absolute change, 7.9%; 95% confidence interval [CI], 2.4 to 13.3). Likewise, TXA use for TKA procedures increased from 62.4 to 82.3% (absolute change, 19.9%; 95% CI 15.0 to 25.0). CONCLUSIONS: Physicians and their teams were able to review their practice data on TXA utilization, reflect on differences compared with evidence-based guidelines, discuss findings with peers, and identify opportunities for improvement. The intervention increased the use of TXA for both TKA and THA and shifted the dosage to better align with evidence-based practice guidelines.

RéSUMé: OBJECTIF : L'acide tranexamique (ATX) peropératoire est utilisé pour réduire les pertes sanguines et les besoins transfusionnels après les arthroplasties totales de la hanche (ATH) et du genou (ATG). Malgré les données probantes et les protocoles de pratique locaux appuyant l'utilisation d'ATX dans le cadre de la norme de soins en cas d'arthroplastie, l'administration de cet agent est sous-utilisée. Notre objectif était d'utiliser l'audit et la rétroaction facilités par le groupe comme base d'une stratégie d'application des connaissances afin d'accroître l'utilisation de l'ATX lors des ATH et ATG. MéTHODE: Les anesthésiologistes ont consenti à recevoir deux rapports de données résumant leurs taux individuels d'utilisation d'ATX et de transfusions sanguines postopératoires par rapport à leurs pairs au sein du même établissement. Les variables recueillies comprenaient les données démographiques des patients, l'utilisation d'ATX et la fréquence et le volume des transfusions d'érythrocytes administrées au cours d'une période postopératoire de 72 heures. La séance de rétroaction facilitée a porté sur les conclusions du rapport et s'est concentrée sur les facteurs contribuant aux habitudes de pratique locales et aux possibilités de changement. RéSULTATS: L'utilisation d'acide tranexamique a augmenté pour les procédures d'ATH au site d'intervention, passant de 66,6 % à 74,4 % (variation absolue, 7,9 %; intervalle de confiance [IC] à 95 %, 2,4 à 13,3). De même, l'utilisation d'ATX pour les procédures d'ATG est passée de 62,4 % à 82,3 % (variation absolue, 19,9 %; IC 95 %, 15,0 à 25,0). CONCLUSION: Les médecins et leurs équipes ont pu passer en revue leurs données de pratique sur l'utilisation d'ATX, réfléchir aux différences par rapport aux lignes directrices fondées sur des données probantes, discuter des résultats avec leurs pairs et identifier les possibilités d'amélioration. L'intervention a augmenté l'utilisation d'ATX pour l'ATG et l'ATH et a modifié la posologie pour mieux s'aligner sur les lignes directrices de pratique fondées sur des données probantes.

Implementation of competence by design in Canadian neurosurgery residency programs.

OBJECTIVE: The Royal College of Physicians and Surgeons of Canada (RCPSC) recently redesigned the Canadian neurosurgery residency training curriculum by implementing a competency-based model of training known as Competence by Design (CBD) centered around the assessment of Entrustable Professional Activities (EPAs). This sequential explanatory mixed-methods study evaluated potential benefits and pitfalls of CBD in Canadian neurosurgery residency education. METHODS: Two four-month interval surveys were distributed to all Canadian neurosurgery residents participating in CBD. The surveys assessed important educational components: CBD knowledge of key stakeholders, potential system barriers, and educational/psychological impacts on residents. Paired t-tests were done to assess changes over time. Based on longitudinal survey responses, semi-structured interviews were conducted to investigate in-depth residents' experience with CBD in neurosurgery. The qualitative analysis followed an explanatory approach, and a thematic analysis was performed. RESULTS: Surveys had 82% average response rate (n = 25). Over time, most residents self-reported that they retrospectively understood concepts around CBD intentions (p = 0.02). Perceived benefits included faculty evaluations with more feedback that was clearer and more objective (53% and 51%). Pitfalls included the amount of time needed to navigate through EPAs (90%) and residents forgetting to initiate EPA forms (71%). There was no significant change over time. During interviews, five key themes were found. Potential solutions identified by residents to enhance their experience included learning analytics data availability, mobile app refinement, and dedicated time to integrate EPAs in the workflow. CONCLUSION: This study was the first to assess resident-perceived benefits and pitfalls of the neurosurgery CBD training program in an educational framework context. In general, residents believed that theoretical principles behind CBD were valuable, but that technological ability and having enough time to request EPA assessments were significant barriers to success. Long-term studies are required to determine the definitive outcomes of CBD on residents' performance and ultimately, on patient care.

Implementation of competence committees during the transition to CBME in Canada: A national fidelity-focused evaluation.

PURPOSE: This study evaluated the fidelity of competence committee (CC) implementation in Canadian postgraduate specialist training programs during the transition to competency-based medical education (CBME). METHODS: A national survey of CC chairs was distributed to all CBME training programs in November 2019. Survey questions were derived from guiding documents published by the Royal College of Physicians and Surgeons of Canada reflecting intended processes and design. RESULTS: Response rate was 39% (113/293) with representation from all eligible disciplines. Committee size ranged from 3 to 20 members, 42% of programs included external members, and 20% included a resident representative. Most programs (72%) reported that a primary review and synthesis of resident assessment data occurs prior to the meeting, with some data reviewed collectively during meetings. When determining entrustable professional activity (EPA) achievement, most programs followed the national specialty guidelines closely with some exceptions (53%). Documented concerns about professionalism, EPA narrative comments, and EPA entrustment scores were most highly weighted when determining resident progress decisions. CONCLUSIONS: Heterogeneity in CC implementation likely reflects local adaptations, but may also explain some of the variable challenges faced by programs during the transition to CBME. Our results offer educational leaders important fidelity data that can help inform the larger evaluation and transformation of CBME.

Ready, set, go! Evaluating readiness to implement competency-based medical education.

PURPOSE: Organizational readiness is critical for successful implementation of an innovation. We evaluated program readiness to implement Competence by Design (CBD), a model of Competency-Based Medical Education (CBME), among Canadian postgraduate training programs. METHODS: A survey of program directors was distributed 1 month prior to CBD implementation in 2019. Questions were informed by the R = MC2 framework of organizational readiness and addressed: program motivation, general capacity for change, and innovation-specific capacity. An overall readiness score was calculated. An ANOVA was conducted to compare overall readiness between disciplines. RESULTS: Survey response rate was 42% (n = 79). The mean overall readiness score was 74% (30-98%). There was no difference in scores between disciplines. The majority of respondents agreed that successful implementation of CBD was a priority (74%), and that their leadership (94%) and faculty and residents (87%) were supportive of change. Fewer perceived that CBD was a move in the right direction (58%) and that implementation was a manageable change (53%). Curriculum mapping, competence committees and programmatic assessment activities were completed by >90% of programs, while <50% had engaged off-service disciplines. CONCLUSION: Our study highlights important areas where programs excelled in their preparation for CBD, as well as common challenges that serve as targets for future intervention to improve program readiness for CBD implementation.

Capturing outcomes of competency-based medical education: The call and the challenge.

There is an urgent need to capture the outcomes of the ongoing global implementation of competency-based medical education (CBME). However, the measurement of downstream outcomes following educational innovations, such as CBME is fraught with challenges stemming from the complexities of medical training, the breadth and variability of inputs, and the difficulties attributing outcomes to specific educational elements. In this article, we present a logic model for CBME to conceptualize an impact pathway relating to CBME and facilitate outcomes evaluation. We further identify six strategies to mitigate the challenges of outcomes measurement: (1) clearly identify the outcome of interest, (2) distinguish between outputs and outcomes, (3) carefully consider attribution versus contribution, (4) connect outcomes to the fidelity and integrity of implementation, (5) pay attention to unanticipated outcomes, and (6) embrace methodological pluralism. Embracing these challenges, we argue that careful and thoughtful evaluation strategies will move us forward in answering the all-important question: Are the desired outcomes of CBME being achieved?

Outcomes of competency-based medical education: A taxonomy for shared language.

As the global transformation of postgraduate medical training continues, there are persistent calls for program evaluation efforts to understand the impact and outcomes of competency-based medical education (CBME) implementation. The measurement of a complex educational intervention such as CBME is challenging because of the multifaceted nature of activities and outcomes. What is needed, therefore, is an organizational taxonomy to both conceptualize and categorize multiple outcomes. In this manuscript we propose a taxonomy that builds on preceding works to organize CBME outcomes across three domains: focus (educational, clinical), level (micro, meso, macro), and timeline (training, transition to practice, practice). We also provide examples of how to conceptualize outcomes of educational interventions across medical specialties using this taxonomy. By proposing a shared language for outcomes of CBME, we hope that this taxonomy will help organize ongoing evaluation work and catalyze those seeking to engage in the evaluation effort to help understand the impact and outcomes of CBME.

Levothyroxine prescribing and laboratory test use after a minor change in reference range for thyroid-stimulating hormone.

BACKGROUND: Prescribing of levothyroxine and rates of thyroid function testing may be sensitive to minor changes in the upper limit of the reference range for thyroid-stimulating hormone (TSH) that increase the proportion of abnormal results. We evaluated the population-level change in levothyroxine prescribing and TSH testing after a minor planned decrease in the upper limit of the reference range for TSH in a large urban centre with a single medical laboratory. METHODS: Using provincial administrative data, we compared predicted volumes of TSH tests with actual TSH test volumes before and after a planned change in the TSH reference range. We also determined the number of new levothyroxine prescriptions for previously untreated patients and the rate of changes to the prescribed dose for those on previously stable, long-term levothyroxine therapy before and after the change in the TSH reference range. RESULTS: Before the change in the TSH reference range, actual and predicted monthly volumes of TSH testing followed an identical course. After the change, actual test volumes exceeded predicted test volumes by 7.3% (95% confidence interval [CI] 5.3%-9.3%) or about 3000 to 5000 extra tests per month. The proportion of patients with newly "abnormal" TSH results almost tripled, from 3.3% (95% CI 3.2%-3.4%) to 9.1% (95% CI 9.0%-9.2%). The rate of new levothyroxine prescriptions increased from 3.24 (95% CI 3.15-3.33) per 1000 population in 2013 to 4.06 (95% CI 3.96-4.15) per 1000 population in 2014. Among patients with preexisting stable levothyroxine therapy, there was a significant increase in the number of dose escalations (p < 0.001) and a total increase of 500 new prescriptions per month. INTERPRETATION: Our findings suggest that clinicians may have responded to mildly elevated TSH results with new or increased levothyroxine prescriptions and more TSH testing. Knowledge translation efforts may be useful to accompany minor changes in reference ranges.

Screening with Papanicolaou tests in Alberta: Are we Choosing Wisely?

OBJECTIVE: To describe the prevalence and geographic distribution of cervical cancer screening, as well as the age groups of those undergoing screening, in Alberta, and to determine if screening practices conform to current guidelines and follow Choosing Wisely Canada recommendations. DESIGN: Descriptive study using data from the Alberta Ministry of Health Analytics and Performance Reporting Branch. SETTING: Alberta. PARTICIPANTS: Women who had 1 or more Papanicolaou tests between 2011 and 2013. MAIN OUTCOME MEASURES: Number of women aged 15 to 20 and those aged 70 and older who had 1 or more Pap tests in a 3-year period; year-to-year trends in screening rates for women in these 2 age groups; trends in screening rates in various geographic regions (ie, cities and zones) in Alberta; and the discipline of clinicians who ordered the Pap tests. RESULTS: Between 2011 and 2013, 805 632 women in the province of Alberta had 1 or more Pap tests for cervical cancer screening. Overall, 25 511 (17.5%) women aged 15 to 20 and 16 818 (10.3%) aged 70 and older were screened contrary to most existing guidelines. Screening rates varied markedly in different geographic regions of the province. Most Pap tests were ordered by family physicians or general practitioners. CONCLUSION: Within the geographic regions of Alberta, provincial, national, and international guidelines for screening with Pap tests are inconsistently followed. This strongly echoes the need for clinicians and patients to consider the Choosing Wisely Canada recommendations and current guidelines for cervical cancer screening.

Neurology Health Advocacy Curriculum: Needs Assessment, Curricular Content and Underlying Components.

BACKGROUND: Lack of a health advocacy curriculum and clarity are obstacles for effectively teaching neurology health advocacy (NHA) to neurology residents. Our purpose is to assess the need and develop content for a NHA curriculum and to describe its underlying components. METHODS: This is a cross-sectional study with two steps. In step one, neurologists and neurology residents at University of Calgary were surveyed about their perception of teaching NHA and asked to rank 56 neurological diseases on a Likert scale based on how well they lend themselves to teaching health advocacy. In step two, curricular items were developed for the top five neurological diseases, using a modified Delphi procedure. The reliability of the survey instrument was determined by Cronbach's alpha. Exploratory factor analysis was used to identify the underlying components of NHA. RESULTS: Forty-six neurologists and 14 neurology residents were surveyed, with a response rate of 88.33%. Fifty-six percent of neurologists and 85% of residents believe that NHA curriculum is needed. The top five neurological presentations, that lend themselves easily to teaching NHA were: stroke/transient ischemic attacks, alcoholism, epilepsy, Alzheimer's disease, and multiple sclerosis. The survey instrument reliability was 0.97. Exploratory factor analysis revealed four factors that can explain the variability in the survey instrument: multidisciplinary approach to neurological disorders, prevention of recurrence of neurological disease, collaboration with other medical subspecialties, and communication with professions outside the medical field. CONCLUSION: Neurologists' and residents' responses support that NHA curriculum is needed. Four components of NHA were identified that can be used for teaching NHA as well as health advocacy in general practice.

Use of an error-focused checklist to identify incompetence in lumbar puncture performances.

CONTEXT: Checklists are commonly used in the assessment of procedural competence. However, on most checklists, high scores are often unable to rule out incompetence as the commission of a few serious procedural errors typically results in only a minimal reduction in performance score. We hypothesised that checklists constructed based on procedural errors may be better at identifying incompetence. OBJECTIVES: This study sought to compare the efficacy of an error-focused checklist and a conventionally constructed checklist in identifying procedural incompetence. METHODS: We constructed a 15-item error-focused checklist for lumbar puncture (LP) based on input from 13 experts in four Canadian academic centres, using a modified Delphi approach, over three rounds of survey. Ratings of 18 video-recorded performances of LP on simulators using the error-focused tool were compared with ratings obtained using a published conventional 21-item checklist. Competence/incompetence decisions were based on global assessment. Diagnostic accuracy was estimated using the area under the curve (AUC) in receiver operating characteristic analyses. RESULTS: The accuracy of the conventional checklist in identifying incompetence was low (AUC 0.11, 95% confidence interval [CI] 0.00-0.28) in comparison with that of the error-focused checklist (AUC 0.85, 95% CI 0.67-1.00). The internal consistency of the error-focused checklist was lower than that of the conventional checklist (α = 0.35 and α = 0.79, respectively). The inter-rater reliability of both tools was high (conventional checklist: intraclass correlation coefficient [ICC] 0.99, 95% CI 0.98-1.00; error-focused checklist: ICC 0.92, 95% CI 0.68-0.98). CONCLUSIONS: Despite higher internal consistency and inter-rater reliability, the conventional checklist was less accurate at identifying procedural incompetence. For assessments in which it is important to identify procedural incompetence, we recommend the use of an error-focused checklist.

Evaluating Competence by Design as a Large System Change Initiative: Readiness, Fidelity, and Outcomes.

Program evaluation is an essential, but often neglected, activity in any transformational educational change. Competence by Design was a large-scale change initiative to implement a competency-based time-variable educational system in Canadian postgraduate medical education. A program evaluation strategy was an integral part of the build and implementation plan for CBD from the beginning, providing insights into implementation progress, challenges, unexpected outcomes, and impact. The Competence by Design program evaluation strategy was built upon a logic model and three pillars of evaluation: readiness to implement, fidelity and integrity of implementation, and outcomes of implementation. The program evaluation strategy harvested from both internally driven studies and those performed by partners and invested others. A dashboard for the program evaluation strategy was created to transparently display a real-time view of Competence by Design implementation and facilitate continuous adaptation and improvement. The findings of the program evaluation for Competence by Design drove changes to all aspects of the Competence by Design implementation, aided engagement of partners, supported change management, and deepened our understanding of the journey required for transformational educational change in a complex national postgraduate medical education system. The program evaluation strategy for Competence by Design provides a framework for program evaluation for any large-scale change in health professions education.

Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Quality evaluation reports: Can a faculty development program make a difference?

BACKGROUND: The quality of medical student and resident clinical evaluation reports submitted by rotation supervisors is a concern. The effectiveness of faculty development (FD) interventions in changing report quality is uncertain. AIMS: This study assessed whether faculty could be trained to complete higher quality reports. METHOD: A 3-h interactive program designed to improve evaluation report quality, previously developed and tested locally, was offered at three different Canadian medical schools. To assess for a change in report quality, three reports completed by each supervisor prior to the workshop and all reports completed for 6 months following the workshop were evaluated by three blinded, independent raters using the Completed Clinical Evaluation Report Rating (CCERR): a validated scale that assesses report quality. RESULTS: A total of 22 supervisors from multiple specialties participated. The mean CCERR score for reports completed after the workshop was significantly higher (21.74 ± 4.91 versus 18.90 ± 5.00, p = 0.02). CONCLUSIONS: This study demonstrates that this FD workshop had a positive impact upon the quality of the participants' evaluation reports suggesting that faculty have the potential to be trained with regards to trainee assessment. This adds to the literature which suggests that FD is an important component in improving assessment quality.

Migraine prevalence, treatment and impact: the canadian women and migraine study.

BACKGROUND: The prevalence of migraine headache varies somewhat across geographic regions. The last Canadian population-based study of migraine was in 1994. We report the findings of the Canadian Women and Migraine Survey. In addition to reporting migraine prevalence in Canadian women, the survey identified current consultation and treatment practices of women with migraine, and the psychological burden of migraine. METHODS: The survey was conducted with a population-based sample of 1210 women using standard telephone research methods. Headache diagnoses were based on the International Headache Society (IHS) Classification. RESULTS: Calculated prevalence of migraine headache was 26%. Only 51% of women with migraine had consulted a physician about their headaches. Women with migraines rely on over-the-counter medications and non-specific prescription medications. Less than 10% of women with migraine use triptans/dihydroergotamine for primary treatment. Ninety seven percent of women with migraine reported at least one psychosocial impact resulting from migraines. CONCLUSIONS: The prevalence of migraine in Canadian women appears static, and is again shown to be slightly higher than that reported in the United States. As in other epidemiologic studies, many women with migraine do not seek medical help for their headaches and perhaps as a result, few are using migraine-specific medications to treat their headaches. The impact of migraine on Canadian women is substantial with almost all women with migraine reporting adverse psychosocial effects of migraines on their lives.

A prospective randomized trial of content expertise versus process expertise in small group teaching.

BACKGROUND: Effective teaching requires an understanding of both what (content knowledge) and how (process knowledge) to teach. While previous studies involving medical students have compared preceptors with greater or lesser content knowledge, it is unclear whether process expertise can compensate for deficient content expertise. Therefore, the objective of our study was to compare the effect of preceptors with process expertise to those with content expertise on medical students' learning outcomes in a structured small group environment. METHODS: One hundred and fifty-one first year medical students were randomized to 11 groups for the small group component of the Cardiovascular-Respiratory course at the University of Calgary. Each group was then block randomized to one of three streams for the entire course: tutoring exclusively by physicians with content expertise (n = 5), tutoring exclusively by physicians with process expertise (n = 3), and tutoring by content experts for 11 sessions and process experts for 10 sessions (n = 3). After each of the 21 small group sessions, students evaluated their preceptors' teaching with a standardized instrument. Students' knowledge acquisition was assessed by an end-of-course multiple choice (EOC-MCQ) examination. RESULTS: Students rated the process experts significantly higher on each of the instrument's 15 items, including the overall rating. Students' mean score (±SD) on the EOC-MCQ exam was 76.1% (8.1) for groups taught by content experts, 78.2% (7.8) for the combination group and 79.5% (9.2) for process expert groups (p = 0.11). By linear regression student performance was higher if they had been taught by process experts (regression coefficient 2.7 [0.1, 5.4], p < .05), but not content experts (p = .09). CONCLUSIONS: When preceptors are physicians, content expertise is not a prerequisite to teach first year medical students within a structured small group environment; preceptors with process expertise result in at least equivalent, if not superior, student outcomes in this setting.

The Calgary Audit and Feedback Framework: a practical, evidence-informed approach for the design and implementation of socially constructed learning interventions using audit and group feedback.

BACKGROUND: Audit and feedback interventions may be strengthened using social interaction. The Calgary office of the Alberta Physician Learning Program (CPLP) developed a process for audit and group feedback for physicians. This paper extends previous work in which we developed a conceptual model of physician responses to audit and group feedback based on a qualitative analysis of six audit and group feedback sessions. The present study explored the mediating factors for successfully engaging physician groups in change planning through audit and group feedback. METHODS: To understand why some groups were more interactive than others, we completed a comparative case analysis of the six audit and group feedback projects from the prior study. We used framework analysis to build the case studies, triangulated our observations across data sources to validate findings, compared the case studies for similarities and differences that influenced social interaction (mediating factors), and thematically categorized mediating factors into an organizing framework. RESULTS: Mediating factors for socially interactive AGFS were a pre-existing relationship between the program team and the physician group, projects addressing important, actionable questions, easily interpretable data visualization in the reports, and facilitation of the groups that included reflective questioning. When these factors were in place (cases 1, 2A, 3), the audit and group feedback sessions were dynamic, with physicians sharing and comparing practices, and raising change cues (such as declaring commitments to de-prescribing, planning educational interventions, and improving documentation). In cases 2C-D, the mediating factors were less well established and in these cases, the sessions showed little physician reflection or change planning. We organized the mediating factors into a framework linking the factors for successful sessions to the conceptual model of physician behaviors which these mediating factors drive. CONCLUSIONS: We propose the Calgary Audit and Feedback Framework as a practical tool to help foster socially constructed learning in audit and group feedback sessions. Ensuring that the four factors, relationship, question choice, data visualization, and facilitation, are considered for design and implementation of audit and group feedback will help physicians move from reactions to their data towards planning for change.