Competence By Design: a transformational national model of time-variable competency-based postgraduate medical education.

Postgraduate medical education is an essential societal enterprise that prepares highly skilled physicians for the health workforce. In recent years, PGME systems have been criticized worldwide for problems with variable graduate abilities, concerns about patient safety, and issues with teaching and assessment methods. In response, competency based medical education approaches, with an emphasis on graduate outcomes, have been proposed as the direction for 21st century health profession education. However, there are few published models of large-scale implementation of these approaches. We describe the rationale and design for a national, time-variable competency-based multi-specialty system for postgraduate medical education called Competence by Design. Fourteen innovations were bundled to create this new system, using the Van Melle Core Components of competency based medical education as the basis for the transformation. The successful execution of this transformational training system shows competency based medical education can be implemented at scale. The lessons learned in the early implementation of Competence by Design can inform competency based medical education innovation efforts across professions worldwide.

Strategies to Enable Transformation in Medical Education: Faculty and Trainee Development in Competence By Design.

Transformative changes in health professions education need to incorporate effective faculty development, but few very large-scale faculty development designs have been described. The Royal College of Physicians and Surgeons of Canada's Competence by Design project was launched to transform the delivery of postgraduate medical education in Canada using a competency-based model. In this paper we outline the goals, principles, and rationale of the Royal College's national strategy for faculty and resident development initiatives to support the implementation of Competence by Design. We describe the activities and resources for both faculty and trainees that facilitated the redesign of training programs for each specialty and subspecialty at the national level, as well as supporting the implementation of the redesign at the local level. This undertaking was not without its challenges: we thus reflect on those challenges, enablers, and the lessons learned, and discuss a continuous quality improvement approach that was taken to iteratively inform the implementation process moving forward.

Frameworks for Integrating Learning Analytics With the Electronic Health Record.

ABSTRACT: The information systems designed to support clinical care have evolved separately from those that support health professions education. This has resulted in a considerable digital divide between patient care and education, one that poorly serves practitioners and organizations, even as learning becomes ever more important to both. In this perspective, we advocate for the enhancement of existing health information systems so that they intentionally facilitate learning. We describe three well-regarded frameworks for learning that can point toward how health care information systems can best evolve to support learning. The Master Adaptive Learner model suggests ways that the individual practitioner can best organize their activities to ensure continual self-improvement. The PDSA cycle similarly proposes actions for improvement but at a health care organization's workflow level. Senge's Five Disciplines of the Learning Organization, a more general framework from the business literature, serves to further inform how disparate information and knowledge flows can be managed for continual improvement. Our main thesis holds that these types of learning frameworks should inform the design and integration of information systems serving the health professions. An underutilized mediator of educational improvement is the ubiquitous electronic health record. The authors list learning analytic opportunities, including potential modifications of learning management systems and the electronic health record, that would enhance health professions education and support the shared goal of delivering high-quality evidence-based health care.

The MedSafer Study-Electronic Decision Support for Deprescribing in Hospitalized Older Adults: A Cluster Randomized Clinical Trial.

IMPORTANCE: Scalable deprescribing interventions may reduce polypharmacy and the use of potentially inappropriate medications (PIMs); however, few studies have been large enough to evaluate the impact that deprescribing may have on adverse drug events (ADEs). OBJECTIVE: To evaluate the effect of an electronic deprescribing decision support tool on ADEs after hospital discharge among older adults with polypharmacy. DESIGN, SETTING, AND PARTICIPANTS: This was a cluster randomized clinical trial of older (≥65 years) hospitalized patients with an expected survival of more than 3 months who were admitted to 1 of 11 acute care hospitals in Canada from August 22, 2017, to January 13, 2020. At admission, participants were taking 5 or more medications per day. Data analyses were performed from January 3, 2021, to September 23, 2021. INTERVENTIONS: Personalized reports of deprescribing opportunities generated by MedSafer software to address usual home medications and measures of prognosis and frailty. Deprescribing reports provided to the treating team were compared with usual care (medication reconciliation). MAIN OUTCOMES AND MEASURES: The primary outcome was a reduction of ADEs within the first 30 days postdischarge (including adverse drug withdrawal events) captured through structured telephone surveys and adjudicated blinded to intervention status. Secondary outcomes were the proportion of patients with 1 or more PIMs deprescribed at discharge and the proportion of patients with an adverse drug withdrawal event (ADWE). RESULTS: A total of 5698 participants (median [range] age, 78 [72-85] years; 2858 [50.2%] women; race and ethnicity data were not collected) were enrolled in 3 clusters and were adjudicated for the primary outcome (control, 3204; intervention, 2494). Despite cluster randomization, there were group imbalances, eg, the participants in the intervention arm were older and had more PIMS prescribed at baseline. After hospital discharge, 4989 (87.6%) participants completed an ADE interview. There was no significant difference in ADEs within 30 days of discharge (138 [5.0%] of 2742 control vs 111 [4.9%] of 2247 intervention participants; adjusted risk difference [aRD] -0.8%; 95% CI, -2.9% to 1.3%). Deprescribing increased from 795 (29.8%) of 2667 control to 1249 (55.4%) of 2256 intervention participants [aRD, 22.2%; 95% CI, 16.9% to 27.4%]. There was no difference in ADWEs between groups. Several post hoc sensitivity analyses, including the use of a nonparametric test to address the low cluster number, group imbalances, and potential biases, did not alter study conclusions. CONCLUSIONS AND RELEVANCE: This cluster randomized clinical trial showed that providing deprescribing clinical decision support during acute hospitalization had no demonstrable impact on ADEs, although the intervention was safe and led to improvements in deprescribing. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03272607.

Curricular needs for training telemedicine physicians: A scoping review.

PURPOSE: The use of telemedicine, a part of 'Virtual Care', is rapidly entering mainstream clinical practice. The ideal curriculum for educating physicians to practice in this emerging field has not been established. We examined the literature to evaluate published curricula for quality and comprehensiveness through the lens of Competency-Based Medical Education (CBME). METHODS: We performed a scoping review using CanMEDS as a framework. Peer-reviewed articles describing telemedicine training curricula were identified. Trainee population, curricular points, stage of implementation, evaluation depth, country, and citations (a marker of quality) were examined. RESULTS: Forty-three curricula from 11 countries were identified, addressing all training levels and covering multiple specialties. Instructional methods included lectures (60.5%), hands-on experiences (76%), directed reading (24%), online modules (21%), reflection (13%), simulation (34%), and group discussions (16%). Hands-on curricula covered all CanMEDS roles more often. Twenty-nine of the implemented curricula were evaluated; 83% were rated positively. CONCLUSIONS: Our scoping review helps inform more comprehensive and efficacious curricula for teaching telemedicine. We suggest centering curricula on a competency-based, outcomes-oriented framework such as CanMEDS with multiple teaching modalities complementing hands-on experiences. This will facilitate rigorous telemedicine training to deliver on the promise of high-quality patient care.

Melatonin Increasingly Used in Hospitalized Patients.

The frequency of melatonin use for insomnia in hospitalized patients is unknown. This study assessed temporal trends of melatonin use in the hospital and compared them with those of use of zopiclone and lorazepam. We performed a retrospective observational study over 6 years from January 2013 to December 2018 at two academic urban hospitals in Toronto, Canada. We abstracted pharmacy dispensing data and standardized rates of medication use by inpatient days. Melatonin use increased from almost none to more than 70 doses per 1,000 inpatient days during 2013-2018, while zopiclone use decreased by 20 doses per 1,000 inpatient days. Melatonin use was twice as high at one hospital and was higher on internal medicine and critical care. Overall use of the three medications increased by 25.7%, which mainly reflects a marked increase in melatonin use. Melatonin is likely being used in a proportion of patients who would not otherwise have received a sleep medication.

Derivation and Validation of a Novel Risk Score to Predict Overcorrection of Severe Hyponatremia: The Severe Hyponatremia Overcorrection Risk (SHOR) Score.

BACKGROUND AND OBJECTIVES: Osmotic demyelination syndrome is the most concerning complication of severe hyponatremia, occurring with an overly rapid rate of serum sodium correction. There are limited clinical tools to aid in identifying individuals at high risk of overcorrection with severe hyponatremia. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: We identified all patients who presented to a tertiary-care hospital emergency department in Ottawa, Canada (catchment area 1.2 million) between January 1, 2003 and December 31, 2015, with serum sodium (corrected for glucose levels) <116 mmol/L. Overcorrection was determined using 14 published criteria. Latent class analysis measured the independent association of baseline factors with a consensus overcorrection status on the basis of the 14 criteria, and was summarized as a risk score, which was validated in two cohorts. RESULTS: A total of 623 patients presented with severe hyponatremia (mean initial value 112 mmol/L; SD 3.2). The prevalence of no, unlikely, possible, and definite overcorrection was 72%, 4%, 10%, and 14%, respectively. Overcorrection was independently associated with decreased level of consciousness (2 points), vomiting (2 points), severe hypokalemia (1 point), hypotonic urine (4 points), volume overload (-5 points), chest tumor (-5 points), patient age (-1 point per decade, over 50 years), and initial sodium level (<110 mmol/L: 4 points; 110-111 mmol/L: 2 points; 112-113 mmol/L: 1 point). These points were summed to create the Severe Hyponatremic Overcorrection Risk (SHOR) score, which was significantly associated with overcorrection status (Spearman correlation 0.45; 95% confidence interval, 0.36 to 0.49) and was discriminating (average dichotomized c-statistic 0.77; 95% confidence interval, 0.73 to 0.81). The internal (n=119) and external (n=95) validation cohorts had significantly greater use of desmopressin, which was significantly associated with the SHOR score. The SHOR score was significantly associated with overcorrection status in the internal (P<0.001) but not external (P=0.39) validation cohort. CONCLUSIONS: In patients presenting with severe hyponatremia, overcorrection was common and predictable using baseline information. Further external validation of the SHOR is required before generalized use.

Diastolic Hypotension May Attenuate Benefits from Intensive Systolic Targets: Secondary Analysis of a Randomized Controlled Trial.

BACKGROUND: The Systolic Blood Pressure Intervention Trial (SPRINT) was a randomized controlled trial that studied 9361 adults ≥50 years of age with systolic blood pressure >130 mm Hg and ≥1 cardiovascular risk factors. Patients were randomized to intensive (≤120 mm Hg) or standard (≤140 mm Hg) systolic targets. In August 2016, a limited dataset was released for secondary analysis. We hypothesized that excessive lowering of diastolic blood pressure could cause harm. Using the data from SPRINT, we sought to determine whether the development of diastolic hypotension during treatment was associated with adverse outcomes. METHODS: We included 8046 patients from SPRINT with a baseline diastolic blood pressure ≥65 mm Hg at study enrollment (4041 intensive target; 4005 standard target). Using Cox proportional hazards models, we evaluated the association between the development of diastolic hypotension (defined as ≤55 mm Hg and modeled as a time-dependent covariate) and the combined outcome of cardiovascular morbidity (myocardial infarction, other acute coronary syndromes, stroke, heart failure) and all-cause death. RESULTS: In multivariable analyses, patients who developed diastolic hypotension had an increased risk for our primary outcome (hazard ratio [HR] 1.67; 95% confidence interval [CI] 1.24-2.26). This was true in both the intensive (HR 1.53; 95% CI, 1.04-2.26) and standard (HR 2.23; 95% CI, 1.40-3.54; P for interaction = .09) treatment arms. CONCLUSIONS: We found an association between diastolic hypotension and the combined endpoint of cardiovascular events and all-cause mortality among SPRINT participants with normal to high diastolic blood pressure at entry. Attention to diastolic blood pressure may be important for optimizing outcomes when targeting systolic blood pressure reduction.

Reduction in Unnecessary Red Blood Cell Folate Testing by Restricting Computerized Physician Order Entry in the Electronic Health Record.

PURPOSE: The red blood cell (RBC) folate test is a laboratory test with limited clinical utility. Previous attempts to reduce physician ordering of unnecessary laboratory tests, including folate levels, have resulted in only modest success. The objective of this study was to assess the effectiveness and impacts of restricting RBC folate ordering in the electronic health record (EHR). METHODS: This was a retrospective observational study that took place from January 2010 to December 2016 at a large academic healthcare network in Toronto, Canada. All inpatients and outpatients who underwent at least 1 RBC folate or vitamin B12 test during the study period were included. Ordering an RBC folate test was restricted to clinicians in gastroenterology and hematology. The option to order the test was removed from other physicians' computerized order entry screens in the EHR in June 2013. RESULTS: RBC folate testing decreased by 94.4% during the study, from a mean of 493.0 ± 48.0 tests per month prior to intervention to 27.6 ± 10.3 tests per month after intervention (P < .001). CONCLUSIONS: Restricting RBC folate ordering in the EHR resulted in a large and sustained reduction in RBC folate testing. Significant cost savings, estimated at more than a quarter of a million Canadian dollars over 3 years, were achieved. There was no significant clinical impact of the intervention on the diagnosis of folate deficiency.

Feedback Credibility in a Formative Postgraduate Objective Structured Clinical Examination: Effects of Examiner Type.

BACKGROUND: Resident perspectives on feedback are key determinants of its acceptance and effectiveness, and provider credibility is a critical element in perspective formation. It is unclear what factors influence a resident's judgment of feedback credibility. OBJECTIVE: We examined how residents perceive the credibility of feedback providers during a formative objective structured clinical examination (OSCE) in 2 ways: (1) ratings of faculty examiners compared with standardized patient (SP) examiners, and (2) ratings of faculty examiners based on alignment of expertise and station content. METHODS: During a formative OSCE, internal medicine residents were randomized to receive immediate feedback from either faculty examiners or SP examiners on communication stations, and at least 1 specialty congruent and either 1 specialty incongruent or general internist faculty examiner for clinical stations. Residents rated perceived credibility of feedback providers on a 7-point scale. Results were analyzed with proportional odds models for ordinal credibility ratings. RESULTS: A total of 192 of 203 residents (95%), 72 faculty, and 10 SPs participated. For communication stations, odds of high credibility ratings were significantly lower for SP than for faculty examiners (odds ratio [OR] = 0.28, P < .001). For clinical stations, credibility odds were lower for specialty incongruent faculty (OR = 0.19, P < .001) and female faculty (OR = 0.45, P < .001). CONCLUSIONS: Faculty examiners were perceived as being more credible than SP examiners, despite standardizing feedback delivery. Specialty incongruency with station content and female sex were associated with lower credibility ratings for faculty examiners.

Cognitive load predicts point-of-care ultrasound simulator performance.

INTRODUCTION: The ability to maintain good performance with low cognitive load is an important marker of expertise. Incorporating cognitive load measurements in the context of simulation training may help to inform judgements of competence. This exploratory study investigated relationships between demographic markers of expertise, cognitive load measures, and simulator performance in the context of point-of-care ultrasonography. METHODS: Twenty-nine medical trainees and clinicians at the University of Toronto with a range of clinical ultrasound experience were recruited. Participants answered a demographic questionnaire then used an ultrasound simulator to perform targeted scanning tasks based on clinical vignettes. Participants were scored on their ability to both acquire and interpret ultrasound images. Cognitive load measures included participant self-report, eye-based physiological indices, and behavioural measures. Data were analyzed using a multilevel linear modelling approach, wherein observations were clustered by participants. RESULTS: Experienced participants outperformed novice participants on ultrasound image acquisition. Ultrasound image interpretation was comparable between the two groups. Ultrasound image acquisition performance was predicted by level of training, prior ultrasound training, and cognitive load. There was significant convergence between cognitive load measurement techniques. A marginal model of ultrasound image acquisition performance including prior ultrasound training and cognitive load as fixed effects provided the best overall fit for the observed data. DISCUSSION: In this proof-of-principle study, the combination of demographic and cognitive load measures provided more sensitive metrics to predict ultrasound simulator performance. Performance assessments which include cognitive load can help differentiate between levels of expertise in simulation environments, and may serve as better predictors of skill transfer to clinical practice.

Outcomes in Severe Hyponatremia Treated With and Without Desmopressin.

BACKGROUND: Overcorrection of plasma sodium in severe hyponatremia is associated with osmotic demyelination syndrome. Desmopressin (DDAVP) can prevent overcorrection of plasma sodium in hyponatremia. The objective of this study was to compare outcomes in hyponatremia according to DDAVP usage. METHODS: This was a retrospective observational study including all admissions to internal medicine with hyponatremia (plasma sodium concentration <123 mEq/L) from 2004 to 2014 at 2 academic hospitals in Toronto, Canada. The primary outcome was safe sodium correction (≤12 mEq/L in any 24-hour and ≤18 mEq/L in any 48-hour period). RESULTS: We identified 1450 admissions with severe hyponatremia; DDAVP was administered in 254 (17.5%). Although DDAVP reduced the rate of change of plasma sodium, fewer patients in the DDAVP group achieved safe correction (174 of 251 [69.3%] vs 970 of 1164 [83.3%]); this result was driven largely by overcorrection occurring before DDAVP administration in the rescue group. Among patients receiving DDAVP, most received it according to a reactive strategy, whereby DDAVP was given after a change in plasma sodium within correction limits (174 of 254 [68.5%]). Suspected osmotic demyelination syndrome was identified in 4 of 1450 admissions (0.28%). There was lower mortality in the DDAVP group (3.9% vs 9.4%), although this is likely affected by confounding. Length of stay in hospital was longer in those who received DDAVP according to a proactive strategy. CONCLUSIONS: Although observational, these data support a reactive strategy for using DDAVP in patients at average risk of osmotic demyelination syndrome, as well as a more stringent plasma sodium correction limit of 8 mEq/L in any 24-hour period for high-risk patients. Further studies are urgently needed on DDAVP use in treating hyponatremia.

Using the Entrustable Professional Activities Framework in the Assessment of Procedural Skills.

BACKGROUND: The entrustable professional activity (EPA) framework has been identified as a useful approach to assessment in competency-based education. To apply an EPA framework for assessment, essential skills necessary for entrustment to occur must first be identified. OBJECTIVE: Using an EPA framework, our study sought to (1) define the essential skills required for entrustment for 7 bedside procedures expected of graduates of Canadian internal medicine (IM) residency programs, and (2) develop rubrics for the assessment of these procedural skills. METHODS: An initial list of essential skills was defined for each procedural EPA by focus groups of experts at 4 academic centers using the nominal group technique. These lists were subsequently vetted by representatives from all Canadian IM training programs through a web-based survey. Consensus (more than 80% agreement) about inclusion of each item was sought using a modified Delphi exercise. Qualitative survey data were analyzed using a framework approach to inform final assessment rubrics for each procedure. RESULTS: Initial lists of essential skills for procedural EPAs ranged from 10 to 24 items. A total of 111 experts completed the national survey. After 2 iterations, consensus was reached on all items. Following qualitative analysis, final rubrics were created, which included 6 to 10 items per procedure. CONCLUSIONS: These EPA-based assessment rubrics represent a national consensus by Canadian IM clinician educators. They provide a practical guide for the assessment of procedural skills in a competency-based education model, and a robust foundation for future research on their implementation and evaluation.

Monitoring and regulation of learning in medical education: the need for predictive cues.

CONTEXT: Being able to accurately monitor learning activities is a key element in self-regulated learning in all settings, including medical schools. Yet students' ability to monitor their progress is often limited, leading to inefficient use of study time. Interventions that improve the accuracy of students' monitoring can optimise self-regulated learning, leading to higher achievement. This paper reviews findings from cognitive psychology and explores potential applications in medical education, as well as areas for future research. COGNITIVE PSYCHOLOGY: Effective monitoring depends on students' ability to generate information ('cues') that accurately reflects their knowledge and skills. The ability of these 'cues' to predict achievement is referred to as 'cue diagnosticity'. Interventions that improve the ability of students to elicit predictive cues typically fall into two categories: (i) self-generation of cues and (ii) generation of cues that is delayed after self-study. Providing feedback and support is useful when cues are predictive but may be too complex to be readily used. APPLICATION TO MEDICAL EDUCATION: Limited evidence exists about interventions to improve the accuracy of self-monitoring among medical students or trainees. Developing interventions that foster use of predictive cues can enhance the accuracy of self-monitoring, thereby improving self-study and clinical reasoning. First, insight should be gained into the characteristics of predictive cues used by medical students and trainees. Next, predictive cue prompts should be designed and tested to improve monitoring and regulation of learning. Finally, the use of predictive cues should be explored in relation to teaching and learning clinical reasoning. CONCLUSIONS: Improving self-regulated learning is important to help medical students and trainees efficiently acquire knowledge and skills necessary for clinical practice. Interventions that help students generate and use predictive cues hold the promise of improved self-regulated learning and achievement. This framework is applicable to learning in several areas, including the development of clinical reasoning.