Association Between Health System Factors and Utilization of Routine Laboratory Tests in Clinical Teaching Units: a Cohort Analysis.

BACKGROUND: Few studies have looked at health system factors associated with laboratory test use. OBJECTIVE: To determine the association between health system factors and routine laboratory test use in medical inpatients. DESIGN: We conducted a retrospective cohort study on adult patients admitted to clinical teaching units over a 3-year period (January 2015 to December 2017) at three tertiary care hospitals in Calgary, Alberta. PARTICIPANTS: Patients were assigned to a Case Mix Group+ (CMG+) category based on their clinical characteristics, and patients in the top 10 CMG+ groups were included in the cohort. EXPOSURES: The examined health system factors were (1) number of primary attending physicians seen by a patient, (2) number of attending medical teams seen by a patient, (3) structure of the medical team, and (4) day of the week. MAIN MEASURES: The primary outcome was the total number of routine laboratory tests ordered on a patient during their admission. Statistical models were adjusted for age, sex, length of stay, Charlson comorbidity index, and CMG+ group. RESULTS: The final cohort consisting of 36,667 patient-days in hospital (mean (SD) age 62.5 (18.4) years) represented 5071 unique hospitalizations and 4324 unique patients. Routine laboratory test use was increased when patients saw multiple attending physicians; with an adjusted incidence rate ratio (IRR) of 1.46 (95% CI, 1.37-1.55) for two attending physicians, and 2.50 (95% CI, 2.23-2.79) for three or more attending physicians compared to a single attending physician. The number of routine laboratory tests was slightly lower on weekends (IRR 0.98, 95% CI, 0.96-0.99) and on teams without a senior resident as part of their team structure (IRR 0.89, 95% CI 0.830.96). CONCLUSIONS: The associations observed in this study suggest that breaks in continuity of care, including increased frequency in patient transfer of care, may impact the utilization of routine laboratory tests.

Lung Ultrasound for Pleural Line Abnormalities, Confluent B-Lines, and Consolidation: Expert Reproducibility and a Method of Standardization.

OBJECTIVES: Discrete B-lines have clear definitions, but confluent B-lines, consolidations, and pleural line abnormalities are less well defined. We proposed definitions for these and determined their reproducibility using COVID-19 patient images obtained with phased array probes. METHODS: Two raters collaborated to refine definitions, analyzing disagreements on 107 derivation scans from 10 patients. Refined definitions were used by those raters and an independent rater on 1260 validation scans from 105 patients. Reliability was evaluated using intraclass correlation coefficients (ICC) or Cohen's kappa. RESULTS: The agreement was excellent between collaborating raters for B-line abnormalities, ICC = 0.97 (95% confidence interval [CI] 0.97-0.98) and pleural line to consolidation abnormalities, ICC = 0.90 (95% CI 0.87-0.92). The independent rater's agreement for B-line abnormalities was excellent, ICC = 0.97 (95% CI 0.96-0.97) and for pleural line to consolidation was good, ICC = 0.88 (95% CI 0.84-0.91). Agreement just on pleural line abnormalities was weak (collaborators, κ = 0.54, 95% CI 0.48-0.60; independent, κ = 0.54, 95% CI 0.49-0.59). CONCLUSION: With proposed definitions or via collaboration, overall agreement on confluent B-lines and pleural line to consolidation abnormalities was robust. Pleural line abnormality agreement itself was persistently weak and caution should be used interpreting pleural line abnormalities with only a phased array probe.

Description of a Multi-faceted COVID-19 Pandemic Physician Workforce Plan at a Multi-site Academic Health System.

BACKGROUND: The evolving COVID-19 pandemic has and continues to present a threat to health system capacity. Rapidly expanding an existing acute care physician workforce is critical to pandemic response planning in large urban academic health systems. INTERVENTION: The Medical Emergency-Pandemic Operations Command (MEOC)-a multi-specialty team of physicians, operational leaders, and support staff within an academic Department of Medicine in Calgary, Canada-partnered with its provincial health system to rapidly develop a comprehensive, scalable pandemic physician workforce plan for non-ventilated inpatients with COVID-19 across multiple hospitals. The MEOC Pandemic Plan comprised seven components, each with unique structure and processes. METHODS: In this manuscript, we describe MEOC's Pandemic Plan that was designed and implemented from March to May 2020 and re-escalated in October 2020. We report on the plan's structure and process, early implementation outcomes, and unforeseen challenges. Data sources included MEOC documents, health system, public health, and physician engagement implementation data. KEY RESULTS: From March 5 to October 26, 2020, 427 patients were admitted to COVID-19 units in Calgary hospitals. In the initial implementation period (March-May 2020), MEOC communications reached over 2500 physicians, leading to 1446 physicians volunteering to provide care on COVID-19 units. Of these, 234 physicians signed up for hospital shifts, and 227 physicians received in-person personal protective equipment simulation training. Ninety-three physicians were deployed on COVID-19 units at four large acute care hospitals. The resurgence of cases in September 2020 has prompted re-escalation including re-activation of COVID-19 units. CONCLUSIONS: MEOC leveraged an academic health system partnership to rapidly design, implement, and refine a comprehensive, scalable COVID-19 acute care physician workforce plan whose components are readily applicable across jurisdictions or healthcare crises. This description may guide other institutions responding to COVID-19 and future health emergencies.

Canadian Internal Medicine Ultrasound (CIMUS) Expert Consensus Statement on the Use of Lung Ultrasound for the Assessment of Medical Inpatients With Known or Suspected Coronavirus Disease 2019.

OBJECTIVES: To develop a consensus statement on the use of lung ultrasound (LUS) in the assessment of symptomatic general medical inpatients with known or suspected coronavirus disease 2019 (COVID-19). METHODS: Our LUS expert panel consisted of 14 multidisciplinary international experts. Experts voted in 3 rounds on the strength of 26 recommendations as "strong," "weak," or "do not recommend." For recommendations that reached consensus for do not recommend, a fourth round was conducted to determine the strength of those recommendations, with 2 additional recommendations considered. RESULTS: Of the 26 recommendations, experts reached consensus on 6 in the first round, 13 in the second, and 7 in the third. Four recommendations were removed because of redundancy. In the fourth round, experts considered 4 recommendations that reached consensus for do not recommend and 2 additional scenarios; consensus was reached for 4 of these. Our final recommendations consist of 24 consensus statements; for 2 of these, the strength of the recommendations did not reach consensus. CONCLUSIONS: In symptomatic medical inpatients with known or suspected COVID-19, we recommend the use of LUS to: (1) support the diagnosis of pneumonitis but not diagnose COVID-19, (2) rule out concerning ultrasound features, (3) monitor patients with a change in the clinical status, and (4) avoid unnecessary additional imaging for patients whose pretest probability of an alternative or superimposed diagnosis is low. We do not recommend the use of LUS to guide admission and discharge decisions. We do not recommend routine serial LUS in patients without a change in their clinical condition.

Pioneering Remotely Piloted Aerial Systems (Drone) Delivery of a Remotely Telementored Ultrasound Capability for Self Diagnosis and Assessment of Vulnerable Populations-the Sky Is the Limit.

Remotely Piloted Aerial Systems (RPAS) are poised to revolutionize healthcare in out-of-hospital settings, either from necessity or practicality, especially for remote locations. RPAS have been successfully used for surveillance, search and rescue, delivery, and equipping drones with telemedical capabilities being considered. However, we know of no previous consideration of RPAS-delivered tele-ultrasound capabilities. Of all imaging technologies, ultrasound is the most portable and capable of providing real-time point-of-care information regarding anatomy, physiology, and procedural guidance. Moreover, remotely guided ultrasound including self-performed has been a backbone of medical care on the International Space Station since construction. The TeleMentored Ultrasound Supported Medical Interventions Group of the University of Calgary partnered with the Southern Alberta Institute of Technology to demonstrate RPAS delivery of a smartphone-supported tele-ultrasound system by the SwissDrones SDO50 RPAS. Upon receipt of the sanitized probe, a completely ultrasound-naïve volunteer was guided by a remote expert located 100 km away using online video conferencing (Zoom), to conduct a self-performed lung ultrasound examination. It proved feasible for the volunteer to examine their anterior chest, sides, and lower back bilaterally, correlating with standard recommended examinations in trauma/critical care, including the critical locations of a detailed COVID-19 lung diagnosis/surveillance examination. We contend that drone-delivered telemedicine including a tele-ultrasound capability could be leveraged to enhance point-of-care diagnostic accuracy in catastrophic emergencies, and allow diagnostic capabilities to be delivered to vulnerable populations in remote locations for whom transport is impractical or undesirable, speeding response times, or obviating the risk of disease transmission depending on the circumstances.

Video Recording in Veterinary Medicine OSCEs: Feasibility and Inter-rater Agreement between Live Performance Examiners and Video Recording Reviewing Examiners.

The Objective Structured Clinical Examination (OSCE) is a valid, reliable assessment of veterinary students' clinical skills that requires significant examiner training and scoring time. This article seeks to investigate the utility of implementing video recording by scoring OSCEs in real-time using live examiners, and afterwards using video examiners from within and outside the learners' home institution. Using checklists, learners (n=33) were assessed by one live examiner and five video examiners on three OSCE stations: suturing, arthrocentesis, and thoracocentesis. When stations were considered collectively, there was no difference between pass/fail outcome between live and video examiners (χ2 = 0.37, p = .55). However, when considered individually, stations (χ2 = 16.64, p < .001) and interaction between station and type of examiner (χ2 = 7.13, p = .03) demonstrated a significant effect on pass/fail outcome. Specifically, learners being assessed on suturing with a video examiner had increased odds of passing the station as compared with their arthrocentesis or thoracocentesis stations. Internal consistency was fair to moderate (0.34-0.45). Inter-rater reliability measures varied but were mostly moderate to strong (0.56-0.82). Video examiners spent longer assessing learners than live raters (mean of 21 min/learner vs. 13 min/learner). Station-specific differences among video examiners may be due to intermittent visibility issues during video capture. Overall, video recording learner performances appears reliable and feasible, although there were time, cost, and technical issues that may limit its routine use.

Correction to: Internal Medicine Point of Care Ultrasound: Indicators It's Here to Stay.

This editorial, "Internal Medicine Point of Care Ultrasound: Indicators It's Here to Stay" (DOI: 10.1007/s11606-019-05268-0), was intended to accompany "Education Indicators for Internal Medicine Point-of-Care Ultrasound: a Consensus Report from the Canadian Internal Medicine Ultrasound (CIMUS) Group".

Multi-organ point-of-care ultrasound for COVID-19 (PoCUS4COVID): international expert consensus.

COVID-19 has caused great devastation in the past year. Multi-organ point-of-care ultrasound (PoCUS) including lung ultrasound (LUS) and focused cardiac ultrasound (FoCUS) as a clinical adjunct has played a significant role in triaging, diagnosis and medical management of COVID-19 patients. The expert panel from 27 countries and 6 continents with considerable experience of direct application of PoCUS on COVID-19 patients presents evidence-based consensus using GRADE methodology for the quality of evidence and an expedited, modified-Delphi process for the strength of expert consensus. The use of ultrasound is suggested in many clinical situations related to respiratory, cardiovascular and thromboembolic aspects of COVID-19, comparing well with other imaging modalities. The limitations due to insufficient data are highlighted as opportunities for future research.

The Canadian Medical Student Ultrasound Curriculum: A Statement From the Canadian Ultrasound Consensus for Undergraduate Medical Education Group.

OBJECTIVES: This study sought to establish by expert review a consensus-based, focused ultrasound curriculum, consisting of a foundational set of focused ultrasound skills that all Canadian medical students would be expected to attain at the end of the medical school program. METHODS: An expert panel of 21 point-of-care ultrasound and educational leaders representing 15 of 17 (88%) Canadian medical schools was formed and participated in a modified Delphi consensus method. Experts anonymously rated 195 curricular elements on their appropriateness to include in a medical school curriculum using a 5-point Likert scale. The group defined consensus as 70% or more experts agreeing to include or exclude an element. We determined a priori that no more than 3 rounds of voting would be performed. RESULTS: Of the 195 curricular elements considered in the first round of voting, the group reached consensus to include 78 and exclude 24. In the second round, consensus was reached to include 4 and exclude 63 elements. In our final round, with 1 additional item added to the survey, the group reached consensus to include an additional 3 and exclude 8 elements. A total of 85 curricular elements reached consensus to be included, with 95 to be excluded. Sixteen elements did not reach consensus to be included or excluded. CONCLUSIONS: By expert opinion-based consensus, the Canadian Ultrasound Consensus for Undergraduate Medical Education Group recommends that 85 curricular elements be considered for inclusion for teaching in the Canadian medical school focused ultrasound curricula.

The Potential for Remotely Mentored Patient-Performed Home Self-Monitoring for New Onset Alveolar-Interstitial Lung Disease.

Purpose: Coronavirus disease 2019 (COVID-19) caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and is an acute respiratory illness. Although most infected persons are asymptomatic or have only mild symptoms, some patients progress to devastating disease; such progression is difficult to predict or identify in a timely manner. COVID-19 patients who do not require hospitalization can self-isolate at home. Calls from one disease epicenter identify the need for homebased isolation with telemedicine surveillance to monitor for impending deterioration. Methodology: Although the dominant approach for these asymptomatic/paucisymptomatic patients is to monitor oxygen saturation, we suggest additionally considering the potential merits and utility of home-based imaging. Chest computed tomography is clearly impractical, but ultrasound has shown comparable sensitivity for lung involvement, with major advantages of short and simple procedures, low cost, and excellent repeatability. Thoracic ultrasound may thus allow remotely identifying the development of pneumonitis at an early stage of illness and potentially averting the risk of insidious deterioration to severe pneumonia and critical illness while in home isolation. Conclusions: Lung sonography can be easily performed by motivated nonmedical caregivers when directed and supervised in real time by experts. Remote mentors could thus efficiently monitor, counsel, and triage multiple home-based patients from their "control center." Authors believe that this approach deserves further attention and study to reduce delays and failures in timely hospitalization of home-isolated patients.

Expert Recommendations on Frequency of Utilization of Common Laboratory Tests in Medical Inpatients: a Canadian Consensus Study.

BACKGROUND: Repetitive inpatient laboratory testing in the face of clinical stability is a marker of low-value care. However, for commonly encountered clinical scenarios on medical units, there are no guidelines defining appropriate use criteria for laboratory tests. OBJECTIVE: This study seeks to establish consensus-based recommendations for the utilization of common laboratory tests in medical inpatients. DESIGN: This study uses a modified Delphi method. Participants completed two rounds of an online survey to determine appropriate testing frequencies for selected laboratory tests in commonly encountered clinical scenarios. Consensus was defined as agreement by at least 80% of participants. PARTICIPANTS: Participants were 36 experts in internal medicine across Canada defined as internists in independent practice for ≥ 5 years with experience in medical education, quality improvement, or both. Experts represented 8 of the 10 Canadian provinces and 13 of 17 academic institutions. MAIN MEASURES: Laboratory tests and clinical scenarios included were those that were considered common on medical units. The final survey contained a total of 45 clinical scenarios looking at the utilization of six laboratory tests (complete blood count, electrolytes, creatinine, urea, international normalized ratio, and partial thromboplastin time). The possible frequency choices were every 2-4 h, 6-8 h, twice a day, daily, every 2-3 days, weekly, or none unless there was specific diagnostic suspicion. These scenarios were reviewed by two internists with training in quality improvement and survey methods. KEY RESULTS: Of the 45 initial clinical scenarios included, we reached consensus on 17 scenarios. We reached weak consensus on an additional 19 scenarios by combining two adjacent frequency categories. CONCLUSIONS: A Canadian expert panel of internists has provided frequency recommendations on the utilization of six common laboratory tests in medical inpatients. These recommendations need validation in prospective studies to assess whether restrictive versus liberal laboratory test ordering impacts patient outcomes.

Education Indicators for Internal Medicine Point-of-Care Ultrasound: a Consensus Report from the Canadian Internal Medicine Ultrasound (CIMUS) Group.

BACKGROUND: Curriculum development and implementation for internal medicine point-of-care ultrasound (IM POCUS) continues to be a challenge for many residency training programs. Education indicators may provide a useful framework to support curriculum development and implementation efforts across programs in order to achieve a consistent high-quality educational experience. OBJECTIVE: This study seeks to establish consensus-based recommendations for education indicators for IM POCUS training programs in Canada. DESIGN: This consensus study uses a modified nominal group technique for voting in the initial round, followed by two additional rounds of online voting, with consensus defined as agreement by at least 80% of the participants. PARTICIPANTS: Participants were 22 leaders with POCUS and/or education expertise from 13 Canadian internal medicine residency programs across 7 provinces. MAIN MEASURES: Education indicators considered were those that related to aspects of the POCUS educational system, could be presented by a single statistical measure, were readily understood, could be reliably measured to provide a benchmark for measuring change, and represented a policy issue. We excluded a priori indicators with low feasibility, are impractical, or assess learner reactions. Candidate indicators were drafted by two academic internists with post-graduate training in POCUS and medical education. These indicators were reviewed by two internists with training in quality improvement prior to presentation to the expert participants. KEY RESULTS: Of the 52 candidate education indicators considered, 6 reached consensus in the first round, 12 in the second, and 4 in the third round. Only 5 indicators reached consensus to be excluded; the remaining indicators did not reach consensus. CONCLUSIONS: The Canadian Internal Medicine Ultrasound (CIMUS) group recommends 22 education indicators be used to guide and monitor internal medicine POCUS curriculum development efforts in Canada.

Integrating point-of-care ultrasound in the ED evaluation of patients presenting with chest pain and shortness of breath.

OBJECTIVE: The differential diagnoses of patients presenting with chest pain (CP) and shortness of breath (SOB) are broad and non-specific. We aimed to 1) determine how use of point-of-care ultrasound (POCUS) impacted emergency physicians' differential diagnosis, and 2) evaluate the accuracy of POCUS when compared to chest radiograph (CXR) and composite final diagnosis. METHODS: We conducted a prospective observational study in a convenience sample of patients presenting with CP and SOB to the Emergency Department (ED). Treating physicians selected possible diagnoses from a pre-indexed list of possible diagnoses of causes of CP and SOB. The final composite diagnosis from a chart review was determined as the reference standard for the diagnosis. The primary analysis involved calculations of sensitivity and specificity for POCUS identifiable diagnoses in detecting cause of CP and SOB. Additional comparative accuracy analysis with CXRs were conducted. RESULTS: 128 patients with a mean age of 64 ±â€¯17 years were included in the study. Using a reference standard of composite final diagnoses, POCUS had equal or higher specificity to CXR for all indications for which it was used, except for pneumonia. POCUS correctly identified all patients with pneumothorax, pleural effusion and pericardial effusion. In patients with a normal thoracic ultrasound, CXR never provided any actionable clinical information. Adding POCUS to the initial evaluation causes a significant narrowing of the differential diagnoses in which the median differential diagnosis from 5 (IQR 3-6) to 3 (IQR 2-4) p < 0.001. CONCLUSION: In evaluation of patients with CP and SOB, POCUS is a highly feasible diagnostic test which can assist in narrowing down the differential diagnoses. In patients with a normal thoracic ultrasound, the added value of a CXR may be minimal.

Comparing Physical Examination With Sonographic Versions of the Same Examination Techniques for Splenomegaly.

OBJECTIVES: To determine whether sonographic versions of physical examination techniques can accurately identify splenomegaly, Castell's method (Ann Intern Med 1967; 67:1265-1267), the sonographic Castell's method, spleen tip palpation, and the sonographic spleen tip technique were compared with reference measurements. METHODS: Two clinicians trained in bedside sonography patients recruited from an urban hematology clinic. Each patient was examined for splenomegaly using conventional percussion and palpation techniques (Castell's method and spleen tip palpation, respectively), as well as the sonographic versions of these maneuvers (sonographic Castell's method and sonographic spleen tip technique). Results were compared with a reference standard based on professional sonographer measurements. RESULTS: The sonographic Castell's method had greater sensitivity (91.7% [95% confidence interval, 61.5% to 99.8%]) than the traditional Castell's method (83.3% [95% confidence interval, 51.6% to 97.9%]) but took longer to perform [mean ± SD, 28.8 ± 18.6 versus 18.8 ± 8.1 seconds; P = .01). Palpable and positive sonographic spleen tip results were both 100% specific, but the sonographic spleen tip method was more sensitive (58.3% [95% confidence interval, 27.7% to 84.8%] versus 33.3% [95% confidence interval, 9.9% to 65.1%]). CONCLUSIONS: Sonographic versions of traditional physical examination maneuvers have greater diagnostic accuracy than the physical examination maneuvers from which they are derived but may take longer to perform. We recommend a combination of traditional physical examination and sonographic techniques when evaluating for splenomegaly at the bedside.

Point of care ultrasound training for internal medicine: a Canadian multi-centre learner needs assessment study.

BACKGROUND: Significant gaps currently exist in the Canadian internal medicine point-of-care ultrasound (POCUS) curriculum. From a learner's perspective, it remains unknown what key POCUS skills should be prioritized. This needs assessment study seeks to establish educational priorities for POCUS for internal medicine residents at five Canadian residency training programs. METHODS: All internal medicine trainees [postgraduate year (PGY) 1-5] from five internal medicine residency training programs in Canada (n = 598) were invited to complete an online survey on 15 diagnostic POCUS applications, 9 bedside procedures, and 18 POCUS knowledge items. For POCUS applications and procedures, participants were asked how applicable they are to patient care in internal medicine and the participants' reported skills in those domains. Self-reported knowledge and skills were rated on a 5-point Likert scale, where 1 = very poor and 5 = very good. Applicability was rated, where 1 = not at all applicable and 5 = very applicable. RESULTS: A total of 253 of 598 residents (42%) participated in our study. Data from one centre (n = 15) was removed because of low response rate (15%) and significant baseline differences between those trainees and the remaining participants. Of the remaining analyzable data from four training programs (n = 238), participants reported highest applicability to internal medicine for the following applications and procedures: identifying ascites/free fluid [mean applicability score of 4.9 ± standard deviation (SD) 0.4]; gross left ventricular function (mean 4.8 ± SD 0.5) and pericardial effusion (mean 4.7 ± SD 0.5); thoracentesis (mean score 4.9 ± SD 0.3), central line insertion (mean 4.9 ± SD 0.3), and paracentesis (mean 4.9 ± SD 0.3), respectively. Overall reported knowledge/skills was low, with skill gaps being the highest for identifying deep vein thrombosis (mean gap 2.7 ± SD 1.1), right ventricular strain (mean 2.7 ± SD 1.1), and gross left ventricular function (mean 2.7 ± SD 1.0). CONCLUSIONS: Many POCUS applications and procedures were felt to be applicable to the practice of internal medicine. Significant skill gaps exist in the four Canadian training programs included in the study. POCUS curriculum development efforts should target training based on these perceived skill gaps.

Internal Medicine Point-of-Care Ultrasound Curriculum: Consensus Recommendations from the Canadian Internal Medicine Ultrasound (CIMUS) Group.

Bedside point-of-care ultrasound (POCUS) is increasingly used to assess medical patients. At present, no consensus exists for what POCUS curriculum is appropriate for internal medicine residency training programs. This document details the consensus-based recommendations by the Canadian Internal Medicine Ultrasound (CIMUS) group, comprising 39 members, representing 14 institutions across Canada. Guiding principles for selecting curricular content were determined a priori. Consensus was defined as agreement by at least 80% of the members on POCUS applications deemed appropriate for teaching and assessment of trainees in the core (internal medicine postgraduate years [PGY] 1-3) and expanded (general internal medicine PGY 4-5) training programs. We recommend four POCUS applications for the core PGY 1-3 curriculum (inferior vena cava, lung B lines, pleural effusion, and abdominal free fluid) and three ultrasound-guided procedures (central venous catheterization, thoracentesis, and paracentesis). For the expanded PGY 4-5 curriculum, we recommend an additional seven applications (internal jugular vein, lung consolidation, pneumothorax, knee effusion, gross left ventricular systolic function, pericardial effusion, and right ventricular strain) and four ultrasound-guided procedures (knee arthrocentesis, arterial line insertion, arterial blood gas sampling, and peripheral venous catheterization). These recommendations will provide a framework for training programs at a national level.

Defining Competencies for Ultrasound-Guided Bedside Procedures: Consensus Opinions From Canadian Physicians.

OBJECTIVES: This study sought to define the competencies in ultrasound knowledge and skills that are essential for medical trainees to master to perform ultrasound-guided central venous catheterization, thoracentesis, and paracentesis. METHODS: Experts in the 3 procedures were identified by a snowball technique through 3 Canadian tertiary academic health centers. Experts completed 2 rounds of surveys, including an 88-item central venous catheterization survey, a 96-item thoracentesis survey, and an 89-item paracentesis survey. For each item, experts were asked to determine whether the knowledge/skill described was essential, important, or marginal. Consensus on an item was defined as agreement by at least 80% of the experts. For items on which consensus was not reached during the first round of surveys, a second survey was created in which the experts were asked to rate the item in a binary fashion (essential/important versus marginal/unimportant). RESULTS: Of the 27 experts invited to complete each survey, 25 (93%) completed the central venous catheterization survey; 22 (81%) completed the thoracentesis survey; and 23 (85%) completed the paracentesis survey. The experts represented 8 specialties from 8 cities within Canada. A total of 22, 32, and 28 items were determined to be essential competencies for central venous catheterization, thoracentesis, and paracentesis, respectively, whereas 47, 38, and 42 competencies were determined to be important, and 8, 13, and 10 were determined to be marginal. The ability to perform real-time direct ultrasound guidance was considered essential only for the performance of central venous catheterization insertion. CONCLUSIONS: Our study presents expert consensus-derived ultrasound competencies that should be considered during the design and implementation of procedural skills training for learners.

A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment.

CONTEXT: The relative advantages and disadvantages of checklists and global rating scales (GRSs) have long been debated. To compare the merits of these scale types, we conducted a systematic review of the validity evidence for checklists and GRSs in the context of simulation-based assessment of health professionals. METHODS: We conducted a systematic review of multiple databases including MEDLINE, EMBASE and Scopus to February 2013. We selected studies that used both a GRS and checklist in the simulation-based assessment of health professionals. Reviewers working in duplicate evaluated five domains of validity evidence, including correlation between scales and reliability. We collected information about raters, instrument characteristics, assessment context, and task. We pooled reliability and correlation coefficients using random-effects meta-analysis. RESULTS: We found 45 studies that used a checklist and GRS in simulation-based assessment. All studies included physicians or physicians in training; one study also included nurse anaesthetists. Topics of assessment included open and laparoscopic surgery (n = 22), endoscopy (n = 8), resuscitation (n = 7) and anaesthesiology (n = 4). The pooled GRS-checklist correlation was 0.76 (95% confidence interval [CI] 0.69-0.81, n = 16 studies). Inter-rater reliability was similar between scales (GRS 0.78, 95% CI 0.71-0.83, n = 23; checklist 0.81, 95% CI 0.75-0.85, n = 21), whereas GRS inter-item reliabilities (0.92, 95% CI 0.84-0.95, n = 6) and inter-station reliabilities (0.80, 95% CI 0.73-0.85, n = 10) were higher than those for checklists (0.66, 95% CI 0-0.84, n = 4 and 0.69, 95% CI 0.56-0.77, n = 10, respectively). Content evidence for GRSs usually referenced previously reported instruments (n = 33), whereas content evidence for checklists usually described expert consensus (n = 26). Checklists and GRSs usually had similar evidence for relations to other variables. CONCLUSIONS: Checklist inter-rater reliability and trainee discrimination were more favourable than suggested in earlier work, but each task requires a separate checklist. Compared with the checklist, the GRS has higher average inter-item and inter-station reliability, can be used across multiple tasks, and may better capture nuanced elements of expertise.