The Ottawa resident observation form for nurses (O-RON): evaluation of an assessment tool's psychometric properties in different specialties.

BACKGROUND: Workplace-based assessment (WBA) used in post-graduate medical education relies on physician supervisors' feedback. However, in a training environment where supervisors are unavailable to assess certain aspects of a resident's performance, nurses are well-positioned to do so. The Ottawa Resident Observation Form for Nurses (O-RON) was developed to capture nurses' assessment of trainee performance and results have demonstrated strong evidence for validity in Orthopedic Surgery. However, different clinical settings may impact a tool's performance. This project studied the use of the O-RON in three different specialties at the University of Ottawa. METHODS: O-RON forms were distributed on Internal Medicine, General Surgery, and Obstetrical wards at the University of Ottawa over nine months. Validity evidence related to quantitative data was collected. Exit interviews with nurse managers were performed and content was thematically analyzed. RESULTS: 179 O-RONs were completed on 30 residents. With four forms per resident, the ORON's reliability was 0.82. Global judgement response and frequency of concerns was correlated (r = 0.627, P < 0.001). CONCLUSIONS: Consistent with the original study, the findings demonstrated strong evidence for validity. However, the number of forms collected was less than expected. Exit interviews identified factors impacting form completion, which included clinical workloads and interprofessional dynamics.

Implicit versus explicit first impressions in performance-based assessment: will raters overcome their first impressions when learner performance changes?

First impressions can influence rater-based judgments but their contribution to rater bias is unclear. Research suggests raters can overcome first impressions in experimental exam contexts with explicit first impressions, but these findings may not generalize to a workplace context with implicit first impressions. The study had two aims. First, to assess if first impressions affect raters' judgments when workplace performance changes. Second, whether explicitly stating these impressions affects subsequent ratings compared to implicitly-formed first impressions. Physician raters viewed six videos where learner performance either changed (Strong to Weak or Weak to Strong) or remained consistent. Raters were assigned two groups. Group one (n = 23, Explicit) made a first impression global rating (FIGR), then scored learners using the Mini-CEX. Group two (n = 22, Implicit) scored learners at the end of the video solely with the Mini-CEX. For the Explicit group, in the Strong to Weak condition, the FIGR (M = 5.94) was higher than the Mini-CEX Global rating (GR) (M = 3.02, p < .001). In the Weak to Strong condition, the FIGR (M = 2.44) was lower than the Mini-CEX GR (M = 3.96 p < .001). There was no difference between the FIGR and the Mini-CEX GR in the consistent condition (M = 6.61, M = 6.65 respectively, p = .84). There were no statistically significant differences in any of the conditions when comparing both groups' Mini-CEX GR. Therefore, raters adjusted their judgments based on the learners' performances. Furthermore, raters who made their first impressions explicit showed similar rater bias to raters who followed a more naturalistic process.

A comparative analysis of graduate preparedness for a career in General Internal Medicine before and after national subspecialty recognition to inform curricular changes: have we met the mark?

Background: A survey of General Internal Medicine (GIM) graduates published in 2006 revealed large training gaps that informed the development of the first national GIM objectives of training in 2010. The first recognized GIM certification examination was written by candidates in 2014. The landscape is again changing with the introduction in 2019 of competency-by-design (CBD) to GIM training. This study aims to examine pre-existing and emerging training gaps with standardization of GIM curricula and identify new training needs to inform CBD curricula. Methods: GIM graduates from all 16 Canadian programs from 2014 -2019 were emailed a survey modeled after the original study published in 2006. Graduates were asked about their preparedness and importance ratings for various elements of practice. Results: Many of the previously identified gaps (difference between importance and preparedness ratings) have been resolved in specific clinical areas (obstetrical and perioperative medicine) and skills (exercise stress testing) although some still require ongoing work in areas such as substance use disorders. Importantly, gaps still exist in preparedness for some intrinsic roles (e.g. managerial skills). Conclusions: The development of a national GIM curriculum has helped close some educational gaps but some still exist. Our study provides data needed to meet the evolving needs of our graduates.

Contexte: Une enquête auprès des diplômés en médecine interne générale (MIG), publiée en 2006, a révélé d'importantes lacunes dans leur formation, menant à l'élaboration des premiers objectifs nationaux de formation en MIG en 2010. Le premier examen de certification en MIG a été organisé en 2014. La formation est à nouveau en train de changer avec l'introduction en 2019 de la compétence par conception (CPC) dans la formation en MIG. Cette étude vise à examiner les lacunes de formation préexistantes et émergentes avec la normalisation de la formation en MIG et à identifier les nouveaux besoins de formation pour éclairer la définition des programmes de formation selon l'approche fondée sur les compétences. Méthodes: Les diplômés des 16 programmes canadiens en MIG entre 2014 et 2019 ont reçu par courriel un sondage inspiré de l'étude originelle publiée en 2006. Les diplômés ont été interrogés sur leur état de préparation et sur l'importance qu'ils accordaient à divers éléments de la pratique. Résultats: Un grand nombre des lacunes décelées précédemment (différence entre les cotes d'importance et de préparation) ont été comblées dans des domaines cliniques spécifiques (médecine obstétrique et périopératoire) et par rapport à des compétences spécifiques (tests de stress à l'effort); dans certains domaines, comme les troubles liés à l'utilisation de substances psychoactives, les efforts doivent être poursuivis. Il est important de noter que des lacunes subsistent dans la préparation à certains rôles intrinsèques (par exemple, les compétences de gestionnaire). Conclusion: L'élaboration d'un programme national de formation en MIG a permis de combler certaines lacunes en matière de formation, mais des carences subsistent. Notre étude fournit les données nécessaires pour répondre aux besoins évolutifs de nos diplômés.

Start spreading the news: a deliberate approach to POCUS program development and implementation.

While there is an expanding body of literature on Point-of-Care Ultrasound (POCUS) pedagogy, administrative elements that are necessary for the widespread adoption of POCUS in the clinical environment have received little attention. In this short communication, we seek to address this gap by sharing our institutional experience with POCUS program development and implementation. The five pillars of our program, selected to tackle local barriers to POCUS uptake, are education, workflow, patient safety, research, and sustainability. Our program logic model outlines the inputs, activities, and outputs of our program. Finally, key indicators for the monitoring of program implementation efforts are presented. Though designed for our local context, this approach may readily be adapted toward other clinical environments. We encourage others leading the integration of POCUS at their centers to adopt this approach not only to achieve sustainable change but also to ensure that quality safeguards are in place.

Knowledge gaps in bleeding disorders: Results from an objective structured oral examination in internal medicine residency training.

BACKGROUND: Empirical evidence for physicians' knowledge gaps in bleeding disorders is limited to self-reported surveys, yet often cited as a leading cause of diagnostic and management delays. OBJECTIVES: Our aim was to assess internal medicine (IM) residents' competence, based on their training level, in evaluating a patient with a bleeding disorder, and knowledge gaps in their clinical approach. METHODS: Content experts developed patient case presenting with abnormal bleeding, bruising, and an isolated prolonged PTT. We administered the hemostasis case as part of an objective structured clinical examination (OSCE). We performed a descriptive analysis. One-way anova was conducted to compare the effect of training level on performance. Item difficulty level for the hemostasis case was also determined. RESULTS: Sixty-seven IM residents participated in the OSCE. The hemostasis case had the highest failure rate at 41.8% with a mean score for the station of 57.96% (SD 13.04). Senior residents scored significantly higher than junior residents on this case (F(2,64) = 4.604, p = .014, ηp2  = 0.126). The item difficulty analysis demonstrated challenges in in eliciting a history of bleeding provoked by challenges, examining the bleeding site, interpreting the mixing study, requesting appropriate follow-up tests, making the diagnosis and providing acute management for a bleeding patient. Only 49.3% of residents requested a hematology consultation. CONCLUSIONS: We demonstrated important knowledge gaps in IM residents' approach to the bleeding patient. Innovative strategies for hemostasis education should be a priority to address physician-related factors in the diagnostic and management delays of patients with bleeding disorders.

Unobserved Observers: Nurses' Perspectives About Sharing Feedback on the Performance of Resident Physicians.

PURPOSE: Postgraduate training programs are incorporating feedback from registered nurses (RNs) to facilitate holistic assessments of resident performance. RNs are a potentially rich source of feedback because they often observe trainees during clinical encounters when physician supervisors are not present. However, RN perspectives about sharing feedback have not been deeply explored. This study investigated RN perspectives about providing feedback and explored the facilitators and barriers influencing their engagement. METHOD: Constructivist grounded theory methodology was used in interviewing 11 emergency medicine and 8 internal medicine RNs at 2 campuses of a tertiary care academic medical center in Ontario, Canada, between July 2019 and March 2020. Interviews explored RN experiences working with and observing residents in clinical practice. Data collection and analysis were conducted iteratively. Themes were identified using constant comparative analysis. RESULTS: RNs felt they could observe authentic day-to-day behaviors of residents often unwitnessed by supervising physicians and offer unique feedback related to patient advocacy, communication, leadership, collaboration, and professionalism. Despite a strong desire to contribute to resident education, RNs were apprehensive about sharing feedback and reported barriers related to hierarchy, power differentials, and a fear of overstepping professional boundaries. Although infrequent, a key stimulus that enabled RNs to feel safe in sharing feedback was an invitation from the supervising physician to provide input. CONCLUSIONS: Perceived hierarchy in academic medicine is a critical barrier to engaging RNs in feedback for residents. Accessing RN feedback on authentic resident behaviors requires dismantling the negative effects of hierarchy and fostering a collaborative interprofessional working environment. A critical step toward this goal may require supervising physicians to model feedback-seeking behavior by inviting RNs to share feedback. Until a workplace culture is established that validates nurses' input and creates safe opportunities for them to contribute to resident education, the voices of nurses will remain unheard.

Post-discharge after surgery Virtual Care with Remote Automated Monitoring-1 (PVC-RAM-1) technology versus standard care: randomised controlled trial.

OBJECTIVE: To determine if virtual care with remote automated monitoring (RAM) technology versus standard care increases days alive at home among adults discharged after non-elective surgery during the covid-19 pandemic. DESIGN: Multicentre randomised controlled trial. SETTING: 8 acute care hospitals in Canada. PARTICIPANTS: 905 adults (≥40 years) who resided in areas with mobile phone coverage and were to be discharged from hospital after non-elective surgery were randomised either to virtual care and RAM (n=451) or to standard care (n=454). 903 participants (99.8%) completed the 31 day follow-up. INTERVENTION: Participants in the experimental group received a tablet computer and RAM technology that measured blood pressure, heart rate, respiratory rate, oxygen saturation, temperature, and body weight. For 30 days the participants took daily biophysical measurements and photographs of their wound and interacted with nurses virtually. Participants in the standard care group received post-hospital discharge management according to the centre's usual care. Patients, healthcare providers, and data collectors were aware of patients' group allocations. Outcome adjudicators were blinded to group allocation. MAIN OUTCOME MEASURES: The primary outcome was days alive at home during 31 days of follow-up. The 12 secondary outcomes included acute hospital care, detection and correction of drug errors, and pain at 7, 15, and 30 days after randomisation. RESULTS: All 905 participants (mean age 63.1 years) were analysed in the groups to which they were randomised. Days alive at home during 31 days of follow-up were 29.7 in the virtual care group and 29.5 in the standard care group: relative risk 1.01 (95% confidence interval 0.99 to 1.02); absolute difference 0.2% (95% confidence interval -0.5% to 0.9%). 99 participants (22.0%) in the virtual care group and 124 (27.3%) in the standard care group required acute hospital care: relative risk 0.80 (0.64 to 1.01); absolute difference 5.3% (-0.3% to 10.9%). More participants in the virtual care group than standard care group had a drug error detected (134 (29.7%) v 25 (5.5%); absolute difference 24.2%, 19.5% to 28.9%) and a drug error corrected (absolute difference 24.4%, 19.9% to 28.9%). Fewer participants in the virtual care group than standard care group reported pain at 7, 15, and 30 days after randomisation: absolute differences 13.9% (7.4% to 20.4%), 11.9% (5.1% to 18.7%), and 9.6% (2.9% to 16.3%), respectively. Beneficial effects proved substantially larger in centres with a higher rate of care escalation. CONCLUSION: Virtual care with RAM shows promise in improving outcomes important to patients and to optimal health system function. TRIAL REGISTRATION: ClinicalTrials.gov NCT04344665.

First year internal medicine residents' self-report point-of-care ultrasound knowledge and skills: what (Little) difference three years make.

BACKGROUND: With increasing availability of point-of-care ultrasound (POCUS) education in medical schools, it is unclear whether or not learning needs of junior medical residents have evolved over time. METHODS: We invited all postgraduate year (PGY)-1 residents at three Canadian internal medicine residency training programs in 2019 to complete a survey previously completed by 47 Canadian Internal Medicine PGY-1 s in 2016. Using a five-point Likert scale, participants rated perceived applicability of POCUS to the practice of internal medicine and self-reported skills in 15 diagnostic POCUS applications and 9 procedures. RESULTS: Of the 97 invited residents, 58 (60 %) completed the survey in 2019. Participants reported high applicability but low skills across all POCUS applications and procedures. The 2019 cohort reported higher skills in assessing pulmonary B lines than the 2016 cohort (2.3 ± SD 1.0 vs. 1.5 ± SD 0.7, adjusted p-value = 0.01). No other differences were noted. CONCLUSIONS: POCUS educational needs continue to be high in Canadian internal medicine learners. The results of this needs assessment study support ongoing inclusion of basic POCUS elements in the current internal medicine residency curriculum.

Implementation of the Canadian Cardiovascular Society guidelines for perioperative risk assessment and management: an interrupted time series study.

PURPOSE: The Canadian Cardiovascular Society (CCS) guidelines for patients undergoing non-cardiac surgery address the lack of standardized management for patients at risk of perioperative cardiovascular complications. Our interdisciplinary group evaluated the implementation of these guidelines. METHODS: We used an interrupted time series design to evaluate the effect of implementation of the CCS guidelines, using routinely collected hospital data. The study population consisted of elective, non-cardiac surgery patients who were: i) inpatients following surgery and ii) age ≥ 65 or age 45-64 yr with a Revised Cardiac Risk Index ≥ 1. Outcomes included adherence to troponin I (TnI) monitoring (primary) and adherence to appropriate consultant care for patients with elevated TnI (secondary). Exploratory outcomes included cost measures and clinical outcomes such as length of stay. RESULTS: We included 1,421 patients (706 pre- and 715 post-implementation). We observed a 67% absolute increase (95% confidence interval, 55 to 80; P < 0.001) in adherence to TnI testing following the implementation of the guidelines. In patients who had elevated TnI following guideline implementation (n = 64), the majority (85%) received appropriate follow-up care in the form of a general medicine or cardiology consult, all received at least one electrocardiogram, and half received at least one advanced cardiac test (e.g., cardiac perfusion scan, or percutaneous intervention). CONCLUSIONS: Our study showed the ability to implement and adhere to the CCS guidelines. Large-scale multicentre evaluations of CCS guideline implementation are needed to gain a better understanding of potential effects on clinically relevant outcomes.

RéSUMé: OBJECTIF: Les lignes directrices de la Société canadienne de cardiologie (SCC) concernant les patients subissant une chirurgie non cardiaque ont été conçues pour pallier l'absence de standardisation dans la prise en charge des patients à risque de complications cardiovasculaires périopératoires. Notre groupe interdisciplinaire a évalué la mise en œuvre de ces lignes directrices. MéTHODE: Nous avons utilisé une méthodologie de série chronologique interrompue pour évaluer l'effet de la mise en œuvre des lignes directrices de la SCC, à l'aide des données hospitalières habituellement recueillies. La population à l'étude se composait de patients de chirurgies non cardiaques non urgentes qui étaient : i) hospitalisés après leur chirurgie et ii) âgés de ≥ 65 ans ou de 45 à 64 ans avec un Indice de risque cardiaque révisé ≥ 1. Les critères d'évaluation comprenaient l'observance du monitorage de la troponine I (TnI) (critère d'évaluation primaire) et l'observance des soins spécialisés appropriés aux patients présentant un taux élevé de TnI (critère secondaire). Les critères exploratoires comprenaient des mesures de coûts et des résultats cliniques tels que la durée de séjour. RéSULTATS: Nous avons inclus 1421 patients (706 avant et 715 après la mise en œuvre). Nous avons observé une augmentation absolue de 67 % (intervalle de confiance de 95 %, 55 à 80; P < 0,001) de l'observance des tests de la TnI suite à la mise en œuvre des lignes directrices. Parmi les patients présentant un taux élevé de TnI suite à la mise en œuvre des lignes directrices (n = 64), la majorité (85%) a reçu des soins de suivi appropriés sous la forme d'une consultation en médecine générale ou en cardiologie; tous ont subi au moins un électrocardiogramme, et la moitié ont passé au moins un examen cardiaque subséquent (p. ex., évaluation de la perfusion myocardique par scintigraphie ou cathétérisme percutané). CONCLUSION: Notre étude a montré qu'il est possible de mettre en œuvre et d'adhérer aux nouvelles lignes directrices de la SCC. Des évaluations multicentriques à grande échelle portant sur la mise en œuvre des lignes directrices de la SCC sont nécessaires pour mieux comprendre ses effets potentiels sur les devenirs cliniquement pertinents.

Post Discharge after Surgery Virtual Care with Remote Automated Monitoring Technology (PVC-RAM): protocol for a randomized controlled trial.

BACKGROUND: After nonelective (i.e., semiurgent, urgent and emergent) surgeries, patients discharged from hospitals are at risk of readmissions, emergency department visits or death. During the coronavirus disease 2019 (COVID-19) pandemic, we are undertaking the Post Discharge after Surgery Virtual Care with Remote Automated Monitoring Technology (PVC-RAM) trial to determine if virtual care with remote automated monitoring (RAM) compared with standard care will increase the number of days adult patients remain alive at home after being discharged following nonelective surgery. METHODS: We are conducting a randomized controlled trial in which 900 adults who are being discharged after nonelective surgery from 8 Canadian hospitals are randomly assigned to receive virtual care with RAM or standard care. Outcome adjudicators are masked to group allocations. Patients in the experimental group learn how to use the study's tablet computer and RAM technology, which will measure their vital signs. For 30 days, patients take daily biophysical measurements and complete a recovery survey. Patients interact with nurses via the cellular modem-enabled tablet, who escalate care to preassigned and available physicians if RAM measurements exceed predetermined thresholds, patients report symptoms, a medication error is identified or the nurses have concerns they cannot resolve. The primary outcome is number of days alive at home during the 30 days after randomization. INTERPRETATION: This trial will inform management of patients after discharge following surgery in the COVID-19 pandemic and offer insights for management of patients who undergo nonelective surgery in a nonpandemic setting. Knowledge dissemination will be supported through an online multimedia resource centre, policy briefs, presentations, peer-reviewed journal publications and media engagement. TRIAL REGISTRATION: ClinicalTrials.gov, no. NCT04344665.

Will I publish this abstract? Determining the characteristics of medical education oral abstracts linked to publication.

BACKGROUND: Prior studies have shown that most conference submissions fail to be published. Understanding factors that facilitate publication may be of benefit to authors. Using data from the Canadian Conference on Medical Education (CCME), our goal was to identify characteristics of conference submissions that predict the likelihood of publication with a specific focus on the utility of peer-review ratings. METHODS: Study characteristics (scholarship type, methodology, population, sites, institutions) from all oral abstracts from 2011-2015 and peer-review ratings for 2014-2015 were extracted by two raters. Publication data was obtained using online database searches. The impact of variables on publication success was analyzed using logistic regressions. RESULTS: In total, 953 oral abstracts were reviewed from 2011 to 2015. Overall, the publication rate was 30.5% (291/953). Of 531 abstracts with peer-review ratings, between 2014 and 2015, 162 (31%) were published. Of the nine analyzed variables, those associated with a greater odds of publication were: multiple vs. single institutions (odds ratio (OR) = 1.72), post-graduate research vs. others (OR=1.81) and peer-review ratings (OR=1.60). Factors with decreased odds of publication were curriculum development (OR=0.17) and innovation vs. others (OR=0.22). CONCLUSION: Similar to other studies, the publication rate of CCME presentations is low. However, peer ratings were predictive of publication success suggesting that ratings could be a useful form of feedback to authors.

CONTEXTE: Des études ont montré que la plupart des résumés soumis pour présentations orales ne sont pas ultérieurement publiés. Il pourrait être utile aux auteurs de comprendre les facteurs qui favorisent la publication. À l'aide de données provenant de la Conférence canadienne sur l'éducation médicale (CCÉM), notre objectif était d'identifier les caractéristiques des résumés permettant de prédire les chances de publication et en particulier l'utilité des cotes attribuées par les réviseurs. MÉTHODOLOGIE: Les caractéristiques des études (type de projet d'érudition, méthodologie, population, établissements, institutions) de tous les résumés de présentation orale soumis pour les conférences de 2011 à 2015 et les cotes attribuées par les réviseurs entre 2014 et 2015 ont été extraites par deux évaluateurs. On a obtenu des données de publication en faisant des recherches dans des bases de données en ligne. L'effet des variables sur le potentiel de publication a été examiné à l'aide de régressions logistiques. RÉSULTATS: Au total, 953 résumés ont été révisé des années 2011 à 2015. Le taux de publication était de 30.5% (291/953) en somme. Des 531 résumés ayant été évalués des pairs, entre 2014 et 2015, 162 (31 %) ont été publiés. Parmi les neuf variables analysées, celles qui ont été associées à un nombre élevé de chances de publication étaient les suivantes : projet multi-institutionnel par rapport à institution unique (risque relatif (RR) = 1,72), travaux de recherche post-graduée par rapport à d'autres types (RR = 1,81) et présence de cotes attribuées par les réviseurs (RR = 1,6). Les facteurs associés à des moindres chances de publication étaient les suivants : articles portant sur le développement de cursus (RR = 0,17) et les innovations, par rapport à d'autres (RR = 0,22). CONCLUSION: Comme ce fut le cas pour d'autres études, le taux de publication à la suite d'une présentation au CCME est faible. Cependant, les cotes attribuées par les réviseurs permettaient de prédire les chances de publication ce qui semble indiquer que les cotes pourraient constituer une forme de rétroaction utile aux auteurs.

Entrustment within an objective structured clinical examination (OSCE) progress test: Bridging the gap towards competency-based medical education.

PURPOSE: Progress testing aligns well with competency-based medical education (CBME) frameworks, which stress the importance of continuous improvement. Entrustment is a useful assessment concept in CBME models. The purpose of this study was to explore the use of an entrustability rating scale within the context of an objective structured clinical examination (OSCE) Progress Test. METHODS: A 9-case OSCE Progress Test was administered to Internal Medicine residents (PGYs 1-4). Residents were assessed using a checklist (CL), global rating scale (GRS), training level rating scale (TLRS), and entrustability scale (ENT). Reliability was calculated using Cronbach's alpha. Differences in performance by training year were explored using ANOVA and effect sizes were calculated using partial eta-squared. Examiners completed a post-examination survey. RESULTS: Ninety one residents and forty two examiners participated in the OSCE. Inter-station reliability was high for all instruments. There was an overall effect of training level for all instruments (p < 0.001). Effect sizes were large. 88% of examiners completed the survey. Most (62%) indicated feeling comfortable in making entrustment decisions during the OSCE. CONCLUSIONS: An entrustability scale can be used in an OSCE Progress Test to generate highly reliable ratings that discriminate between learners at different levels of training.

Correction to: Internal Medicine Point of Care Ultrasound: Indicators It's Here to Stay.

This editorial, "Internal Medicine Point of Care Ultrasound: Indicators It's Here to Stay" (DOI: 10.1007/s11606-019-05268-0), was intended to accompany "Education Indicators for Internal Medicine Point-of-Care Ultrasound: a Consensus Report from the Canadian Internal Medicine Ultrasound (CIMUS) Group".

Avoid reinventing the wheel: implementation of the Ottawa Clinic Assessment Tool (OCAT) in Internal Medicine.

BACKGROUND: Workplace based assessment (WBA) is crucial to competency-based education. The majority of healthcare is delivered in the ambulatory setting making the ability to run an entire clinic a crucial core competency for Internal Medicine (IM) trainees. Current WBA tools used in IM do not allow a thorough assessment of this skill. Further, most tools are not aligned with the way clinical assessors conceptualize performances. To address this, many tools aligned with entrustment decisions have recently been published. The Ottawa Clinic Assessment Tool (OCAT) is an entrustment-aligned tool that allows for such an assessment but was developed in the surgical setting and it is not known if it can perform well in an entirely different context. The aim of this study was to implement the OCAT in an IM program and collect psychometric data in this different setting. Using one tool across multiple contexts may reduce the need for tool development and ensure that tools used have proper psychometric data to support them. METHODS: Psychometrics characteristics were determined. Descriptive statistics and effect sizes were calculated. Scores were compared between levels of training (juniors (PGY1), seniors (PGY2s and PGY3s) & fellows (PGY4s and PGY5s)) using a one-way ANOVA. Safety for independent practice was analyzed with a dichotomous score. Variance components were generated and used to estimate the reliability of the OCAT. RESULTS: Three hundred ninety OCATs were completed over 52 weeks by 86 physicians assessing 44 residents. The range of ratings varied from 2 (I had to talk them through) to 5 (I did not need to be there) for most items. Mean scores differed significantly by training level (p < .001) with juniors having lower ratings (M = 3.80 (out of 5), SD = 0.49) than seniors (M = 4.22, SD = - 0.47) who had lower ratings than fellows (4.70, SD = 0.36). Trainees deemed safe to run the clinic independently had significantly higher mean scores than those deemed not safe (p < .001). The generalizability coefficient that corresponds to internal consistency is 0.92. CONCLUSIONS: This study's psychometric data demonstrates that we can reliably use the OCAT in IM. We support assessing existing tools within different contexts rather than continuous developing discipline-specific instruments.

Point of care ultrasound training for internal medicine: a Canadian multi-centre learner needs assessment study.

BACKGROUND: Significant gaps currently exist in the Canadian internal medicine point-of-care ultrasound (POCUS) curriculum. From a learner's perspective, it remains unknown what key POCUS skills should be prioritized. This needs assessment study seeks to establish educational priorities for POCUS for internal medicine residents at five Canadian residency training programs. METHODS: All internal medicine trainees [postgraduate year (PGY) 1-5] from five internal medicine residency training programs in Canada (n = 598) were invited to complete an online survey on 15 diagnostic POCUS applications, 9 bedside procedures, and 18 POCUS knowledge items. For POCUS applications and procedures, participants were asked how applicable they are to patient care in internal medicine and the participants' reported skills in those domains. Self-reported knowledge and skills were rated on a 5-point Likert scale, where 1 = very poor and 5 = very good. Applicability was rated, where 1 = not at all applicable and 5 = very applicable. RESULTS: A total of 253 of 598 residents (42%) participated in our study. Data from one centre (n = 15) was removed because of low response rate (15%) and significant baseline differences between those trainees and the remaining participants. Of the remaining analyzable data from four training programs (n = 238), participants reported highest applicability to internal medicine for the following applications and procedures: identifying ascites/free fluid [mean applicability score of 4.9 ± standard deviation (SD) 0.4]; gross left ventricular function (mean 4.8 ± SD 0.5) and pericardial effusion (mean 4.7 ± SD 0.5); thoracentesis (mean score 4.9 ± SD 0.3), central line insertion (mean 4.9 ± SD 0.3), and paracentesis (mean 4.9 ± SD 0.3), respectively. Overall reported knowledge/skills was low, with skill gaps being the highest for identifying deep vein thrombosis (mean gap 2.7 ± SD 1.1), right ventricular strain (mean 2.7 ± SD 1.1), and gross left ventricular function (mean 2.7 ± SD 1.0). CONCLUSIONS: Many POCUS applications and procedures were felt to be applicable to the practice of internal medicine. Significant skill gaps exist in the four Canadian training programs included in the study. POCUS curriculum development efforts should target training based on these perceived skill gaps.

Transitioning towards senior medical resident: identification of the required competencies using consensus methodology.

BACKGROUND: Residency programs are facing significant restructuring through the "Competence by Design" (CBD) framework proposed by the Royal College of Physicians and Surgeons of Canada (RCPSC). Our goal was to establish the competencies to be acquired during the transition to a senior role within Internal Medicine (IM) training. METHODS: Using a modified Delphi technique, practicing IM physicians and recent graduates were polled to develop consensus on the required competencies to effectively transition from junior to senior medical resident. Participants rated each competency on a three-point Likert scale. Each competency was linked to an Entrustable Professional Activity (EPA) identified by the RCPSC IM Specialty Committee. RESULTS: A total of eighteen participants took part in item generation (16% response rate) and nineteen in the initial ranking with seventeen completing all three iterations (89% completion rate). Eighty-three competencies were identified during questionnaire development. A final list of seventy-seven competencies reached consensus after three rounds. Most competencies matched to core of discipline EPAs. CONCLUSION: This consensus-based list of competencies will help create a framework and tools for the assessment of junior residents as they prepare to transition to the role of senior in the new CBD curricula for IM trainees at our institution.

The implementation and evaluation of an e-Learning training module for objective structured clinical examination raters in Canada.

Improving the reliability and consistency of objective structured clinical examination (OSCE) raters' marking poses a continual challenge in medical education. The purpose of this study was to evaluate an e-Learning training module for OSCE raters who participated in the assessment of third-year medical students at the University of Ottawa, Canada. The effects of online training and those of traditional in-person (face-to-face) orientation were compared. Of the 90 physicians recruited as raters for this OSCE, 60 consented to participate (67.7%) in the study in March 2017. Of the 60 participants, 55 rated students during the OSCE, while the remaining 5 were back-up raters. The number of raters in the online training group was 41, while that in the traditional in-person training group was 19. Of those with prior OSCE experience (n= 18) who participated in the online group, 13 (68%) reported that they preferred this format to the in-person orientation. The total average time needed to complete the online module was 15 minutes. Furthermore, 89% of the participants felt the module provided clarity in the rater training process. There was no significant difference in the number of missing ratings based on the type of orientation that raters received. Our study indicates that online OSCE rater training is comparable to traditional face-to-face orientation.

Internal Medicine Point-of-Care Ultrasound Curriculum: Consensus Recommendations from the Canadian Internal Medicine Ultrasound (CIMUS) Group.

Bedside point-of-care ultrasound (POCUS) is increasingly used to assess medical patients. At present, no consensus exists for what POCUS curriculum is appropriate for internal medicine residency training programs. This document details the consensus-based recommendations by the Canadian Internal Medicine Ultrasound (CIMUS) group, comprising 39 members, representing 14 institutions across Canada. Guiding principles for selecting curricular content were determined a priori. Consensus was defined as agreement by at least 80% of the members on POCUS applications deemed appropriate for teaching and assessment of trainees in the core (internal medicine postgraduate years [PGY] 1-3) and expanded (general internal medicine PGY 4-5) training programs. We recommend four POCUS applications for the core PGY 1-3 curriculum (inferior vena cava, lung B lines, pleural effusion, and abdominal free fluid) and three ultrasound-guided procedures (central venous catheterization, thoracentesis, and paracentesis). For the expanded PGY 4-5 curriculum, we recommend an additional seven applications (internal jugular vein, lung consolidation, pneumothorax, knee effusion, gross left ventricular systolic function, pericardial effusion, and right ventricular strain) and four ultrasound-guided procedures (knee arthrocentesis, arterial line insertion, arterial blood gas sampling, and peripheral venous catheterization). These recommendations will provide a framework for training programs at a national level.