Design and Implementation of a National Program of Assessment Model - Integrating Entrustable Professional Activity Assessments in Canadian Specialist Postgraduate Medical Education.

Traditional approaches to assessment in health professions education systems, which have generally focused on the summative function of assessment through the development and episodic use of individual high-stakes examinations, may no longer be appropriate in an era of competency based medical education. Contemporary assessment programs should not only ensure collection of high-quality performance data to support robust decision-making on learners' achievement and competence development but also facilitate the provision of meaningful feedback to learners to support reflective practice and performance improvement. Programmatic assessment is a specific approach to designing assessment systems through the intentional selection and combination of a variety of assessment methods and activities embedded within an educational framework to simultaneously optimize the decision-making and learning function of assessment. It is a core component of competency based medical education and is aligned with the goals of promoting assessment for learning and coaching learners to achieve predefined levels of competence. In Canada, postgraduate specialist medical education has undergone a transformative change to a competency based model centred around entrustable professional activities (EPAs). In this paper, we describe and reflect on the large scale, national implementation of a program of assessment model designed to guide learning and ensure that robust data is collected to support defensible decisions about EPA achievement and progress through training. Reflecting on the design and implications of this assessment system may help others who want to incorporate a competency based approach in their own country.

Current use of simulation for EPA assessment in emergency medicine.

OBJECTIVE: Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment. METHODS: A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views. RESULTS: A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment. CONCLUSIONS: Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

RéSUMé: OBJECTIF: Il y a environ cinq ans, les programmes de médecine d'urgence du Collège royal au Canada ont mis en place un paradigme basé sur les compétences et ont introduit l'utilisation d'activités professionnelles confiables (APC) pour l'évaluation des unités d'activité professionnelle afin d'évaluer les stagiaires. De nombreux programmes d'enseignement médical basés sur les compétences (CBME) prévoient l'évaluation des compétences par l'observation des APC. Bien que les APC soient fréquemment évaluées en milieu clinique, la simulation est également utilisée. Cette étude visait à caractériser l'utilisation de la simulation pour l'évaluation de l'APC. MéTHODES: Un guide d'entretien d'étude a été élaboré conjointement par tous les auteurs de l'étude et a suivi les meilleures pratiques en matière d'élaboration d'enquêtes. Un entretien national a été réalisé avec les directeurs de programmes ou les directeurs adjoints de tous les programmes de médecine d'urgence du Collège royal au Canada. Les entretiens ont été menés sur Microsoft Teams, enregistrés et transcrits à l'aide du service de transcription de Microsoft Teams. Les transcriptions des échantillons ont été analysées pour développer des thèmes. Les thèmes ont ensuite été revus par les co-auteurs pour s'assurer qu'ils étaient représentatifs des points de vue des participants. RéSULTATS: Un taux de réponse de 64,7 % a été obtenu. La simulation a été largement adoptée par les programmes de formation en médecine d'urgence. Toutes les personnes interrogées se sont montrées favorables à l'utilisation de la simulation pour l'évaluation de l'APE pour de nombreuses raisons. Cependant, les DP ont reconnu des limites et l'analyse thématique a révélé certains thèmes et tensions liés à l'utilisation de la simulation pour l'évaluation de l'APC. L'analyse thématique a révélé six thèmes majeurs : un appui généralisé à l'utilisation de la simulation pour l'évaluation de l'APC, inquiétudes concernant la possibilité que l'évaluation de l'APC devienne un exercice de type « case à cocher ¼, des obstacles logistiques limitant l'utilisation de la simulation pour l'évaluation de l'APC, les perceptions variées quant à l'authenticité de l'utilisation de la simulation pour l'évaluation de l'APC, le potentiel de la simulation pour l'évaluation de l'APC de compromettre la sécurité psychologique des apprenants, et des suggestions pour l'optimisation de l'utilisation de la simulation pour l'évaluation de l'APC. CONCLUSIONS: Nos résultats offrent un aperçu à d'autres programmes et spécialités sur la meilleure façon d'utiliser la simulation pour l'évaluation de l'APC. Les programmes devraient utiliser ces résultats lorsqu'ils envisagent d'utiliser la simulation pour l'évaluation de l'APC.

Common variable immunodeficiency disorder (CVID)-related liver disease: assessment of the main histological aspects using novel semiquantitative scoring systems, image analysis and correlation with clinical parameters of liver stiffness and portal hypertension.

AIMS: We aimed to investigate the relationship between T-cell-mediated sinusoidal injury, nodular regenerative hyperplasia like changes (NRH-LC) and fibrosis, clinical measures of fibrosis and portal hypertension, and progression rate in common variable immunodeficiency disorder (CVID)-related liver disease. METHODS: This is a retrospective single-centre study. Liver biopsies from CVID patients with liver disease were reviewed to assess for NRH-LC, fibrosis and elastosis, including collagen and elastin proportionate areas. CD3 positive T-cells infiltration and sinusoidal endothelial changes by CD34 expression were quantified by image analysis and a semiquantitative method, respectively. These findings were correlated with liver stiffness measurements (LSM) and hepatic venous pressure gradient (HVPG). RESULTS: NRH-LC and pericellular elastosis were present in most biopsies (32/40 and 38/40, respectively). All biopsies showed fibrosis, which was limited to pericellular in 21/40 (52.5%) and included bridging fibrous septa in 19/40 (47.5%). 28/40 liver biopsies showed enhanced sinusoidal expression of CD34. There were more CD3 positive cells in biopsies with NRH-LC compared with those without. There was no significant correlation between LSM, HVPG and fibrosis/elastosis scores. Five of seven patients with at least two biopsies showed progression in fibrosis stage. CONCLUSIONS: NRH-LC and fibrosis in CVID patients often coexist along with the presence of sinusoidal endothelial changes and sinusoidal lymphocytic infiltration. Fibrosis progresses over time, and significant fibrosis can be observed in young patients (<30 years old), potentially reflecting a more aggressive form of CVID-related liver disease. Further studies are necessary to investigate the relationship between histological findings, clinical measures of fibrosis and portal hypertension and outcome.

Simulation vs workplace-based assessment in resuscitation: a cross-specialty descriptive analysis and comparison.

Background: Simulation-based assessment can complement workplace-based assessment of rare or difficult to assess Entrustable Professional Activities (EPAs). We aimed to compare the use of simulation-based assessment for resuscitation-focused EPAs in three postgraduate medical training programs and describe faculty perceptions of simulation-based assessment. Methods: EPA assessment scores and setting (simulation or workplace) were extracted from 2017-2020 for internal medicine, emergency medicine, and surgical foundations residents at the transition to discipline and foundations of discipline stages. A questionnaire was distributed to clinical competency committee members. Results: Eleven percent of EPA assessments were simulation-based. The proportion of simulation-based assessment did not differ between programs but differed between transition (38%) and foundations (4%) stages within surgical foundations only. Entrustment scores differed between settings in emergency medicine at the transition level only (simulation: 4.82 ± 0.60 workplace: 3.74 ± 0.93). 70% of committee members (n=20) completed the questionnaire. Of those that use simulation-based assessment, 45% interpret them differently than workplace-based assessments. 73% and 100% trust simulation for high-stakes and low-stakes assessment, respectively. Conclusions: The proportion of simulation-based assessment for resuscitation focused EPAs did not differ between three postgraduate medical training programs. Interpretation of simulation-based assessment data between committee members was inconsistent. All respondents trust simulation-based assessment for low-stakes, and the majority for high-stakes assessment. These findings have practical implications for the integration simulation into programs of assessment.

Contexte: Pour les activités professionnelles confiables (APC) qui sont rarement observées ou difficiles à évaluer, une évaluation en séance de simulation peut compléter celle en milieu de travail. Nous avons comparé le recours à une évaluation en séance de simulation pour les APC axées sur la réanimation dans trois programmes de formation médicale postdoctorale et décrit les perceptions de membres du corps professoral à propos de cette modalité d'évaluation. Méthodes: Nous avons extrait les scores et le cadre (simulation ou lieu de travail) d'évaluation des APC de 2017 à 2020 pour les résidents en médecine interne, en médecine d'urgence et en fondements chirurgicaux aux étapes de transition vers la discipline et de fondements de la discipline. Un questionnaire a été distribué aux membres des comités des compétences cliniques. Résultats: Onze pour cent des évaluations d'APC étaient faites lors de séances de simulation. Cette proportion était la même pour tous les programmes, mais dans le cadre des fondements chirurgicaux elle était différente selon qu'il s'agissait de l'étape de transition (38 %) ou de celle des fondements (4 %). Les scores de confiance différaient selon le cadre de l'évaluation uniquement pour les résidents en médecine d'urgence à l'étape de la transition (simulation : 4,82 ± 0,60; lieu de travail : 3,74 ± 0,93). Le questionnaire a été rempli par 70 % des membres des comités (n=20). Parmi ceux qui avaient eu recours à une évaluation en séance de simulation, 45 % avaient interprété les données de l'évaluation différemment de la façon dont ils interprètent les données d'évaluation en milieu de travail. Soixante-treize pour cent et 100 % d'entre eux font confiance à la simulation pour les évaluations à enjeux élevés et à faibles enjeux, respectivement. Conclusions: La proportion d'évaluations en séance de simulation pour les APC axées sur la réanimation était la même dans trois programmes de formation médicale postdoctorale. Les membres des comités de compétences cliniques n'ont pas interprété les données de ce type d'évaluation de manière uniforme. Tous les répondants font confiance à l'évaluation en séance de simulation pour les évaluations à faibles enjeux, et la plupart d'entre eux pour les évaluations à enjeux élevés. Ces données ont des implications pratiques pour l'intégration de la simulation dans les programmes d'évaluation.

Artificial Intelligence Applications for Assessment, Monitoring, and Management of Parkinson Disease Symptoms: Protocol for a Systematic Review.

BACKGROUND: Parkinson disease (PD) is the second most prevalent neurodegenerative disease, with around 10 million people with PD worldwide. Current assessments of PD symptoms are conducted by questionnaires and clinician assessments and have many limitations, including unreliable reporting of symptoms, little autonomy for patients over their disease management, and standard clinical review intervals regardless of disease status or clinical need. To address these limitations, digital technologies including wearable sensors, smartphone apps, and artificial intelligence (AI) methods have been implemented for this population. Many reviews have explored the use of AI in the diagnosis of PD and management of specific symptoms; however, there is limited research on the application of AI to the monitoring and management of the range of PD symptoms. A comprehensive review of the application of AI methods is necessary to address the gap of high-quality reviews and highlight the developments of the use of AI within PD care. OBJECTIVE: The purpose of this protocol is to guide a systematic review to identify and summarize the current applications of AI applied to the assessment, monitoring, and management of PD symptoms. METHODS: This review protocol was structured using the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols) and the Population, Intervention, Comparator, Outcome, and Study (PICOS) frameworks. The following 5 databases will be systematically searched: PubMed, IEEE Xplore, Institute for Scientific Information's Web of Science, Scopus, and the Cochrane Library. Title and abstract screening, full-text review, and data extraction will be conducted by 2 independent reviewers. Data will be extracted into a predetermined form, and any disagreements in screening or extraction will be discussed. Risk of bias will be assessed using the Cochrane Collaboration Risk of Bias 2 tool for randomized trials and the Mixed Methods Appraisal Tool for nonrandomized trials. RESULTS: As of April 2023, this systematic review has not yet been started. It is expected to begin in May 2023, with the aim to complete by September 2023. CONCLUSIONS: The systematic review subsequently conducted as a product of this protocol will provide an overview of the AI methods being used for the assessment, monitoring, and management of PD symptoms. This will identify areas for further research in which AI methods can be applied to the assessment or management of PD symptoms and could support the future implementation of AI-based tools for the effective management of PD. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/46581.

The Assessment Burden in Competency-Based Medical Education: How Programs Are Adapting.

Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter's experience, this article describes postgraduate programs' adaptations related to the challenges of assessment in CBME. From June 2019-September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution's experience and navigate the CBME-related assessment burden their invested partners may be facing.

The Reliability of the Resuscitation Assessment Tool (RAT) in Assessing Emergency Medicine Resident Competence in Pediatric Resuscitation Scenarios: A Prospective Observational Pilot Study.

Introduction Emergency medicine (EM) postgraduate medical education in Canada has transitioned from traditional time-based training to competency-based medical education (CBME). In order to promote residents through stages of training, simulated assessments are needed to evaluate residents in high-stakes but low-frequency medical emergencies. There remains a gap in the literature pertaining to the use of evaluative tools in simulation, such as the Resuscitation Assessment Tool (RAT) in the new CBME curriculum design. Methods We completed a pilot study of resident physicians in one Canadian EM training program to evaluate the effectiveness and reliability of a simulation-based RAT for pediatric resuscitation. We recorded 10 EM trainees completing simulated scenarios and had nine EM physicians use the RAT tool to evaluate their performances. Generalizability theory was used to evaluate the reliability of the RAT tool. Results The mean RAT score for the management of pediatric myocarditis, cardiac arrest, and septic shock (appendicitis) across raters was 3.70, 3.73, and 4.50, respectively. The overall generalizability coefficient for testing simulated pediatric performance competency was 0.77 for internal consistency and 0.75 for absolute agreement. The performance of senior participants was superior to that of junior participants in the management of pediatric myocarditis (p = 0.01) but not statistically significant in the management of pediatric septic shock (p=0.77) or cardiac arrest (p =0.61). Conclusion Overall, our findings suggest that with an appropriately chosen simulated scenario, the RAT tool can be used effectively for the simulation of high-stakes and low-frequency scenarios for practice to enhance the new CBME curriculum in emergency medicine training programs.

The Future of Emergency Medicine (EM) Sim Cases: A Modified Massive Online Needs Assessment.

Objective Emergency Medicine (EM) Sim Cases was initially developed in 2015 as a free open-access simulation resource. To ensure the future of EM Sim Cases remains relevant and up to date, we performed a needs assessment to better define our audience and facilitate long-term goals. Methods We delivered a survey using a modified massive-online-needs-assessment methodology through an iterative process with simulation experts from the EM Simulation Educators Research Collaborative. We distributed the survey via email and Twitter and analyzed the data using descriptive statistics and thematic analysis. Results We obtained 106 responses. EM Sim Cases is commonly used by physicians primarily as an educational resource for postgraduate level trainees. Perceived needs included resuscitation, pediatrics, trauma, and toxicology content. Prompted needs included non-simulation-case educational resources, increased case database, and improved website organization. Conclusions Data collected from our needs assessment has defined our audience allowing us to design our long-term goals and strategies.

Patient, parental and multi-disciplinary team rationale for non-implantation following the paediatric cochlear implantation assessment.

OBJECTIVES: To assess paediatric patients referred to the cochlear implant team, who despite undergoing the assessment, did not receive cochlear implantation. To identify the underlying reasons for this. METHODS: A retrospective case-note review of patients was assessed from June 2014 to June 2019 at two separate London teaching hospitals with paediatric cochlear implantation programmes. RESULTS: A total of 921 paediatric patients were assessed during the study period across both institutions. And, 196 (21%) did not proceed with the surgery. The decision not to undergo surgery was primarily parental/patient-directed in 114 (61%) and cochlear implant team-directed in 74 (39%). In total, eight (4%) patients exited the programme without a documented reason. DISCUSSION: A myriad of factors influenced the decision-making process for clinicians, patients and parents. The most cited parental/patient rationale against implantation was the wish to continue current means of communication 40 (35%), followed by concern regarding the risks of surgery 18 (15.8%) and the wish to allow the patient to make future decisions independently in view of the future technology 7 (6.1%). Cochlear implant team-directed decisions were largely due to being outside of the NICE criteria 27 (36.5%) or the risks of general anaesthesia 19 (25.7%) in addition to communication concerns 11 (14.9%). CONCLUSION: Decision-making in cochlear implantation should not be underestimated. Extensive discussion and exploration of options with the multi-disciplinary team can aid decision-making, but the timescale and appreciation of the consequences of the decision inevitably lead to pressure. Exploration of reasons for non-implantation emphasises the importance of a multi-professional approach to manage these patients.

The ultrasound competency assessment tool for four-view cardiac POCUS.

BACKGROUND: Point-of-care ultrasound (POCUS) has been recognized as an essential skill across medicine. However, a lack of reliable and streamlined POCUS assessment tools with demonstrated validity remains a significant barrier to widespread clinical integration. The ultrasound competency assessment tool (UCAT) was derived to be a simple, entrustment-based competency assessment tool applicable to multiple POCUS applications. When used to assess a FAST, the UCAT demonstrated high internal consistency and moderate-to-excellent inter-rater reliability. The objective of this study was to validate the UCAT for assessment of a four-view transthoracic cardiac POCUS. RESULTS: Twenty-two trainees performed a four-view transthoracic cardiac POCUS in a simulated environment while being assessed by two observers. When used to assess a four-view cardiac POCUS the UCAT retained its high internal consistency ([Formula: see text] and moderate-to-excellent inter-rater reliability (ICCs = 0.61-0.91; p's ≤ 0.01) across all domains. The regression analysis suggestion that level of training, previous number of focused cardiac ultrasound, previous number of total scans, self-rated entrustment, and intent to pursue certification statistically significantly predicted UCAT entrustment scores [F (5,16) = 4.06, p = 0.01; R2 = 0.56]. CONCLUSION: This study confirms the UCAT is a valid assessment tool for four-view transthoracic cardiac POCUS. The findings from this work and previous studies on the UCAT demonstrate the utility and flexibility of the UCAT tool across multiple POCUS applications and present a promising way forward for POCUS competency assessment.

Exploring the use of rating scales with entrustment anchors in workplace-based assessment.

PURPOSE: Competency-based medical education (CBME) has prompted widespread implementation of workplace-based assessment (WBA) tools using entrustment anchors. This study aimed to identify factors that influence faculty's rating choices immediately following assessment and explore their experiences using WBAs with entrustment anchors, specifically the Ottawa Surgical Competency Operating Room Evaluation scale. METHOD: A convenience sample of 50 semi-structured interviews with Emergency Medicine (EM) physicians from a single Canadian hospital were conducted between July and August 2019. All interviews occurred within two hours of faculty completing a WBA of a trainee. Faculty were asked what they considered when rating the trainee's performance and whether they considered an alternate rating. Two team members independently analysed interview transcripts using conventional content analysis with line-by-line coding to identify themes. RESULTS: Interviews captured interactions between 70% (26/37) of full-time EM faculty and 86% (19/22) of EM trainees. Faculty most commonly identified the amount of guidance the trainee required as influencing their rating. Other variables such as clinical context, trainee experience, past experiences with the trainee, perceived competence and confidence were also identified. While most faculty did not struggle to assign ratings, some had difficulty interpreting the language of entrustment anchors, being unsure whether their assessment should be retrospective or prospective in nature, and if/how the assessment should change whether they were 'in the room' or not. CONCLUSIONS: By going to the frontline during WBA encounters, this study captured authentic and honest reflections from physicians immediately engaged in assessment using entrustment anchors. While many of the factors identified are consistent with previous retrospective work, we highlight how some faculty consider factors outside the prescribed approach and struggle with the language of entrustment anchors. These results further our understanding of 'in-the-moment' assessments using entrustment anchors and may facilitate effective faculty development regarding WBA in CBME.

Key considerations in planning and designing programmatic assessment in competency-based medical education.

Programmatic assessment as a concept is still novel for many in clinical education, and there may be a disconnect between the academics who publish about programmatic assessment and the front-line clinical educators who must put theory into practice. In this paper, we clearly define programmatic assessment and present high-level guidelines about its implementation in competency-based medical education (CBME) programs. The guidelines are informed by literature and by lessons learned from established programmatic assessment approaches. We articulate five steps to consider when implementing programmatic assessment in CBME contexts: articulate the purpose of the program of assessment, determine what must be assessed, choose tools fit for purpose, consider the stakes of assessments, and define processes for interpreting assessment data. In the process, we seek to offer a helpful guide or template for front-line clinical educators. We dispel some myths about programmatic assessment to help training programs as they look to design-or redesign-programs of assessment. In particular, we highlight the notion that programmatic assessment is not 'one size fits all'; rather, it is a system of assessment that results when shared common principles are considered and applied by individual programs as they plan and design their own bespoke model of programmatic assessment for CBME in their unique context.

The Ultrasound Competency Assessment Tool (UCAT): Development and Evaluation of a Novel Competency-based Assessment Tool for Point-of-care Ultrasound.

OBJECTIVES: Point-of-care ultrasound (POCUS) has become an integral diagnostic and interventional tool. Barriers to POCUS training persist, and it continues to remain heterogeneous across training programs. Structured POCUS assessment tools exist, but remain limited in their feasibility, acceptability, reliability, and validity; none of these tools are entrustment-based. The objective of this study was to derive a simple, entrustment-based POCUS competency assessment tool and pilot it in an assessment setting. METHODS: This study was composed of two phases. First, a three-step modified Delphi design surveyed 60 members of the Canadian Association of Emergency Physicians Emergency Ultrasound Committee (EUC) to derive the anchors for the tool. Subsequently, the derived ultrasound competency assessment tool (UCAT) was used to assess trainee (N = 37) performance on a simulated FAST examination. The intraclass correlation (ICC) for inter-rater reliability and Cronbach's alpha for internal consistency were calculated. A statistical analysis was performed to compare the UCAT to other competency surrogates. RESULTS: The three-round Delphi had 22, 26, and 26 responses from the EUC members. Consensus was reached, and anchors for the domains of preparation, image acquisition, image optimization, and clinical integration achieved approval rates between 92 and 96%. The UCAT pilot revealed excellent inter-rater reliability (with ICC values of 0.69-0.89; p < 0.01) and high internal consistency (α = 0.91). While UCAT scores were not impacted by level of training, they were significantly impacted by the number of previous POCUS studies completed. CONCLUSIONS: We developed and successfully piloted the UCAT, an entrustment-based bedside POCUS competency assessment tool suitable for rapid deployment. The findings from this study indicate early validity evidence for the use of the UCAT as an assessment of trainee POCUS competence on FAST. The UCAT should be trialed in different populations performing several POCUS study types.

Becoming a deliberately developmental organization: Using competency based assessment data for organizational development.

Medical education is situated within health care and educational organizations that frequently lag in their use of data to learn, develop, and improve performance. How might we leverage competency-based medical education (CBME) assessment data at the individual, program, and system levels, with the goal of redefining CBME from an initiative that supports the development of physicians to one that also fosters the development of the faculty, administrators, and programs within our organizations? In this paper we review the Deliberately Developmental Organization (DDO) framework proposed by Robert Kegan and Lisa Lahey, a theoretical framework that explains how organizations can foster the development of their people. We then describe the DDO's conceptual alignment with CBME and outline how CBME assessment data could be used to spur the transformation of health care and educational organizations into digitally integrated DDOs. A DDO-oriented use of CBME assessment data will require intentional investment into both the digitalization of assessment data and the development of the people within our organizations. By reframing CBME in this light, we hope that educational and health care leaders will see their investments in CBME as an opportunity to spur the evolution of a developmental culture.

Cost and benefit of military quarantine policies.

The initial response to COVID-19 included quarantine policies. This study aims to determine the infection containment proportions and cost of two variations of quarantine policies based on geographic travel and close contact with infected individuals within deployed US military populations. Special Operations Command Africa (SOCAF) records of individuals quarantined between March 1, 2020 and June 1, 2020 were examined. The infection containment proportion and cost in containment hours were compared between types of quarantine and between geographic areas. Geographic quarantine contained 2 cases out of 63 quarantined individuals in West Africa (3.2%) compared to 0 out of 221 in East Africa (p = 0.0486). Close contact quarantine contained 3 cases out of 31 quarantined individuals in West Africa compared to 4 out of 55 in East Africa (7.3%, p = 0.6989). Total confinement was 42,048 h for each contained infection using geographic quarantine compared to 4076 h using close contact quarantine. In the US military population deployed to Africa for COVID-19, quarantining based on geographic movement is an order of magnitude more costly in terms of time for each contained infection then quarantining based on close contact with infected individuals. There is not a statistical difference between East and West Africa. The associated costs of quarantine must be carefully weighed against the risk of disease spread.

Endoscopic Multipoint Laser System for Objective Pediatric Airway Assessment.

Recent technological advances within aeronautical engineering have demonstrated the delivery of objective quantitative endoscopic measurements to within one-hundredth of a millimeter. We sought to validate this emerging laser technology in a simulation-based assessment of pediatric airway stenosis. A 4.4-mm flexible endoscope, incorporating a laser measurement system projecting 49 laser points into the endoscopic view, was used to assess a simulated model of subglottic stenosis. Multiple anteroposterior and lateral measurements were obtained for each stenosis and compared with standard airway assessment techniques. Intra- and interobserver reliability was assessed. A total of 240 multipoint laser measurements were obtained of simulated airway stenosis. The mean difference from manual measurement was 0.1886 mm. The Bland-Altman plot showed low bias (0.011) and narrow 95% limits of agreement (-0.46 to 0.48). This advanced endoscopic measurement technique shows great promise for clinical development to benefit ongoing assessment and treatment of evolving pediatric airway stenosis.

Articulating the effect of food systems innovation on the Sustainable Development Goals.

Food system innovations will be instrumental to achieving multiple Sustainable Development Goals (SDGs). However, major innovation breakthroughs can trigger profound and disruptive changes, leading to simultaneous and interlinked reconfigurations of multiple parts of the global food system. The emergence of new technologies or social solutions, therefore, have very different impact profiles, with favourable consequences for some SDGs and unintended adverse side-effects for others. Stand-alone innovations seldom achieve positive outcomes over multiple sustainability dimensions. Instead, they should be embedded as part of systemic changes that facilitate the implementation of the SDGs. Emerging trade-offs need to be intentionally addressed to achieve true sustainability, particularly those involving social aspects like inequality in its many forms, social justice, and strong institutions, which remain challenging. Trade-offs with undesirable consequences are manageable through the development of well planned transition pathways, careful monitoring of key indicators, and through the implementation of transparent science targets at the local level.

Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study.

BACKGROUND: In 2018, Canadian postgraduate emergency medicine (EM) programs began implementing a competency-based medical education (CBME) assessment program. Studies evaluating these programs have focused on broad outcomes using data from national bodies and lack data to support program-specific improvement. OBJECTIVE: We evaluated the implementation of a CBME assessment program within and across programs to identify successes and opportunities for improvement at the local and national levels. METHODS: Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of entrustable professional activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages were compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. RESULTS: Data from 15 of 17 (88%) programs containing 9842 EPA observations from 68 of 77 (88%) EM residents in the 2018 cohort were analyzed. Average numbers of EPAs observed per resident in each program varied from 92.5 to 229.6, correlating with the number of blocks spent on EM and pediatric EM (r = 0.83, P < .001). Relative to the specialty committee's guidelines, residents were promoted later than expected (eg, one-third of residents had a 2-month delay to promotion from the first to second stage) and with fewer EPA observations than suggested. CONCLUSIONS: There was demonstrable variation in EPA-based assessment numbers and promotion timelines between programs and with national guidelines.

Harnessing the power of simulation for assessment: Consensus recommendations for the use of simulation-based assessment in emergency medicine.

OBJECTIVES: To address the increasing demand for the use of simulation for assessment, our objective was to review the literature pertaining to simulation-based assessment and develop a set of consensus-based expert-informed recommendations on the use of simulation-based assessment as presented at the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium on Education. METHODS: A panel of Emergency Medicine (EM) physicians from across Canada, with leadership roles in simulation and/or assessment, was formed to develop the recommendations. An initial scoping literature review was conducted to extract principles of simulation-based assessment. These principles were refined via thematic analysis, and then used to derive a set of recommendations for the use of simulation-based assessment, organized by the Consensus Framework for Good Assessment. This was reviewed and revised via a national stakeholder survey, and then the recommendations were presented and revised at the consensus conference to generate a final set of recommendations on the use of simulation-based assessment in EM. CONCLUSION: We developed a set of recommendations for simulation-based assessment, using consensus-based expert-informed methods, across the domains of validity, reproducibility, feasibility, educational and catalytic effects, acceptability, and programmatic assessment. While the precise role of simulation-based assessment will be a subject of continued debate, we propose that these recommendations be used to assist educators and program leaders as they incorporate simulation-based assessment into their programs of assessment.