Evaluating the Reliability of a Remote Acuity Prediction Tool in a Canadian Academic Emergency Department.

STUDY OBJECTIVE: There is increasing interest in harnessing artificial intelligence to virtually triage patients seeking care. The objective was to examine the reliability of a virtual machine learning algorithm to remotely predict acuity scores for patients seeking emergency department (ED) care by applying the algorithm to retrospective ED data. METHODS: This was a retrospective review of adult patients conducted at an academic tertiary care ED (annual census 65,000) from January 2021 to August 2022. Data including ED visit date and time, patient age, sex, reason for visit, presenting complaint and patient-reported pain score were used by the machine learning algorithm to predict acuity scores. The algorithm was designed to up-triage high-risk complaints to promote safety for remote use. The predicted scores were then compared to nurse-led triage scores previously derived in real time using the electronic Canadian Triage and Acuity Scale (eCTAS), an electronic triage decision-support tool used in the ED. Interrater reliability was estimated using kappa statistics with 95% confidence intervals (CIs). RESULTS: In total, 21,469 unique ED patient encounters were included. Exact modal agreement was achieved for 10,396 (48.4%) patient encounters. Interrater reliability ranged from poor to fair, as estimated using unweighted kappa (0.18, 95% CI 0.17 to 0.19), linear-weighted kappa (0.25, 95% CI 0.24 to 0.26), and quadratic-weighted kappa (0.36, 95% CI 0.35 to 0.37) statistics. Using the nurse-led eCTAS score as the reference, the machine learning algorithm overtriaged 9,897 (46.1%) and undertriaged 1,176 (5.5%) cases. Some of the presenting complaints under-triaged were conditions generally requiring further probing to delineate their nature, including abnormal lab/imaging results, visual disturbance, and fever. CONCLUSION: This machine learning algorithm needs further refinement before being safely implemented for patient use.

Cost Evaluation of the Ontario Virtual Urgent Care Pilot Program: Population-Based, Matched Cohort Study.

BACKGROUND: In 2020, the Ministry of Health (MoH) in Ontario, Canada, introduced a virtual urgent care (VUC) pilot program to provide alternative access to urgent care services and reduce the need for in-person emergency department (ED) visits for patients with low acuity health concerns. OBJECTIVE: This study aims to compare the 30-day costs associated with VUC and in-person ED encounters from an MoH perspective. METHODS: Using administrative data from Ontario (the most populous province of Canada), a population-based, matched cohort study of Ontarians who used VUC services from December 2020 to September 2021 was conducted. As it was expected that VUC and in-person ED users would be different, two cohorts of VUC users were defined: (1) those who were promptly referred to an ED by a VUC provider and subsequently presented to an ED within 72 hours (these patients were matched to in-person ED users with any discharge disposition) and (2) those seen by a VUC provider with no referral to an in-person ED (these patients were matched to patients who presented in-person to the ED and were discharged home by the ED physician). Bootstrap techniques were used to compare the 30-day mean costs of VUC (operational costs to set up the VUC program plus health care expenditures) versus in-person ED care (health care expenditures) from an MoH perspective. All costs are expressed in Canadian dollars (a currency exchange rate of CAD $1=US $0.76 is applicable). RESULTS: We matched 2129 patients who presented to an ED within 72 hours of VUC referral and 14,179 patients seen by a VUC provider without a referral to an ED. Our matched populations represented 99% (2129/2150) of eligible VUC patients referred to the ED by their VUC provider and 98% (14,179/14,498) of eligible VUC patients not referred to the ED by their VUC provider. Compared to matched in-person ED patients, 30-day costs per patient were significantly higher for the cohort of VUC patients who presented to an ED within 72 hours of VUC referral ($2805 vs $2299; difference of $506, 95% CI $139-$885) and significantly lower for the VUC cohort of patients who did not require ED referral ($907 vs $1270; difference of $362, 95% CI 284-$446). Overall, the absolute 30-day costs associated with the 2 VUC cohorts were $18.9 million (ie, $6.0 million + $12.9 million) versus $22.9 million ($4.9 million + $18.0 million) for the 2 in-person ED cohorts. CONCLUSIONS: This costing evaluation supports the use of VUC as most complaints were addressed without referral to ED. Future research should evaluate targeted applications of VUC (eg, VUC models led by nurse practitioners or physician assistants with support from ED physicians) to inform future resource allocation and policy decisions.

Health care utilization and outcomes of patients seen by virtual urgent care versus in-person emergency department care.

BACKGROUND: Virtual urgent care (VUC) is intended to support diversion of patients with low-acuity complaints and reduce the need for in-person emergency department visits. We aimed to describe subsequent health care utilization and outcomes of patients who used VUC compared with similar patients who had an in-person emergency department visit. METHODS: We used patient-level encounter data that were prospectively collected for patients using VUC services provided by 14 pilot programs in Ontario, Canada. We linked the data to provincial administrative databases to identify subsequent 30-day health care utilization and outcomes. We defined 2 subgroups of VUC users; those with a documented prompt referral to an emergency department by a VUC provider, and those without. We matched patients in each cohort to an equal number of patients presenting to an emergency department in person, based on encounter date, medical concern and the logit of a propensity score. For the subgroup of patients not promptly referred to an emergency department, we matched patients to those who were seen in an emergency department and then discharged home. RESULTS: Of the 19 595 patient VUC visits linked to administrative data, we matched 2129 patients promptly referred to the emergency department by a VUC provider to patients presenting to the emergency department in person. Index visit hospital admissions (9.4% v. 8.7%), 30-day emergency department visits (17.0% v. 17.5%), and hospital admissions (12.9% v. 11.0%) were similar between the groups. We matched 14 179 patients who were seen by a VUC provider with no documented referral to the emergency department. Patients seen by VUC were more likely to have a subsequent in-person emergency department visit within 72 hours (13.7% v. 7.0%), 7 days (16.5% v. 10.3%) and 30 days (21.9% v. 17.9%), but hospital admissions were similar within 72 hours (1.1% v. 1.3%), and higher within 30 days for patients who were discharged home from the emergency department (2.6% v. 3.4%). INTERPRETATION: The impact of the provincial VUC pilot program on subsequent health care utilization was limited. There is a need to better understand the inherent limitations of virtual care and ensure future virtual providers have timely access to in-person outpatient resources, to prevent subsequent emergency department visits.

Development of a machine learning-based acuity score prediction model for virtual care settings.

OBJECTIVE: Healthcare is increasingly digitized, yet remote and automated machine learning (ML) triage prediction systems for virtual urgent care use remain limited. The Canadian Triage and Acuity Scale (CTAS) is the gold standard triage tool for in-person care in Canada. The current work describes the development of a ML-based acuity score modelled after the CTAS system. METHODS: The ML-based acuity score model was developed using 2,460,109 de-identified patient-level encounter records from three large healthcare organizations (Ontario, Canada). Data included presenting complaint, clinical modifiers, age, sex, and self-reported pain. 2,041,987 records were high acuity (CTAS 1-3) and 416,870 records were low acuity (CTAS 4-5). Five models were trained: decision tree, k-nearest neighbors, random forest, gradient boosting regressor, and neural net. The outcome variable of interest was the acuity score predicted by the ML system compared to the CTAS score assigned by the triage nurse. RESULTS: Gradient boosting regressor demonstrated the greatest prediction accuracy. This final model was tuned toward up triaging to minimize patient risk if adopted into the clinical context. The algorithm predicted the same score in 47.4% of cases, and the same or more acute score in 95.0% of cases. CONCLUSIONS: The ML algorithm shows reasonable predictive accuracy and high predictive safety and was developed using the largest dataset of its kind to date. Future work will involve conducting a pilot study to validate and prospectively assess reliability of the ML algorithm to assign acuity scores remotely.

Diversion of hospital admissions from the emergency department using an interprofessional team: a propensity score analysis.

PURPOSE: To examine if an ED interprofessional team ("ED1Team") could safely decrease hospital admissions among older persons. METHODS: This single-center, retrospective, propensity score matched study was performed at a single ED during a control (December 2/2018-March 31/2019) and intervention (December 2/2019-March 31/2020) period. The intervention was assessed by the ED1Team, which could include an occupational therapist, physiotherapist, and social worker. We compared admission rates between period in persons age ≥ 70 years. Next, we compared visits attended by the ED1Team to (a) control period visits, and (b) intervention period visits without ED1Team attendance. SECONDARY OUTCOMES: ED length-of-stay, 7-day subsequent hospital admission and mortality in discharged patients. RESULTS: There were 5496 and 4876 eligible ED visits during the control and intervention periods, respectively. In the latter group, 556 (11.4%) received ED1Team assessment. After matching, there was an absolute 2.3% (p = 0.07) reduction in the admission rate between control and intervention periods. After matching the 556 ED1Team attended visits to control period visits, and to intervention period visits without the intervention, admission rates decreased by 10.0% (p = 0.006) and 13.5% (p < 0.001), respectively. For discharged patients, median ED length-of-stay decreased by 1.0 h (p < 0.001) between control and intervention periods and increased by 2.3 h (p < 0.001) compared to intervention period without the intervention. For patients discharged by the ED1Team, subsequent readmissions after 7 days were slightly higher, but mortality was not significantly different. CONCLUSION: ED1Team consultation was associated with a decreased hospital admission rate in older ED patients. It was associated with a slightly longer ED length-of-stay and subsequent early hospitalizations. Given that even a small increase in freed hospital beds would release some of the pressure on an overextended healthcare system, these results suggest that upscaling of the intervention might procure systems-wide benefits.

RéSUMé: OBJECTIF: Examiner si une équipe interprofessionnelle de DE (« ED1Team ¼) pourrait réduire en toute sécurité les admissions à l'hôpital chez les personnes âgées. MéTHODES: Cette étude rétrospective, à un seul centre et correspondant au score de propension a été réalisée à un seul DE pendant une période de contrôle (2/2018-31 mars/2019) et d'intervention (2/2019-31 mars/2020). L'intervention était une évaluation par l'équipe de l'ED1, qui pouvait comprendre un ergothérapeute, un physiothérapeute et un travailleur social. Nous avons comparé les taux d'admission entre périodes chez des personnes âgées de 70 ans. Ensuite, nous avons comparé les visites auxquelles a assisté l'équipe de DE1 à des visites pendant la période de contrôle et b) des visites pendant la période d'intervention sans présence de l'équipe. Résultats secondaires Durée du séjour en salle d'opération, hospitalisation subséquente de 7 jours et mortalité chez les patients libérés. RéSULTATS: Il y a eu 5496 et 4876 visites admissibles à la DE pendant les périodes de contrôle et d'intervention, respectivement. Dans ce dernier groupe, 556 (11,4 %) ont reçu une évaluation de l'équipe ED1. Après appariement, il y a eu une réduction absolue de 2,3 % (p=0,07) du taux d'admission entre les périodes de contrôle et d'intervention. Après avoir comparé les 556 visites de l'équipe ED1P aux visites des périodes de contrôle et aux visites des périodes d'intervention sans intervention, les taux d'admission ont diminué de 10,0 % (p=0,006) et 13,5 % (p<0,001), respectivement. Chez les patients ayant reçu leur congé, la durée médiane de séjour en DE a diminué de 1,0 heure (p<0,001) entre les périodes de contrôle et d'intervention et a augmenté de 2,3 heures (p<0,001) par rapport à la période d'intervention sans l'intervention. Pour les patients libérés par l'équipe ED1, les réadmissions subséquentes après 7 jours étaient légèrement plus élevées, mais la mortalité n'était pas significativement différente. CONCLUSION: La consultation d'Ed1Team a été associée à une diminution du taux d'admission hospitalière chez les patients âgés atteints de DE. Elle était associée à une durée de séjour légèrement plus longue en salle d'opération et à des hospitalisations précoces subséquentes. Étant donné que même une petite augmentation du nombre de lits d'hôpitaux libérés allégerait en partie la pression exercée sur un système de soins de santé surdimensionné, ces résultats suggèrent qu'une mise à l'échelle de l'intervention pourrait procurer des avantages à l'échelle du système.

A qualitative study on the Virtual Emergency Department care experiences of equity-deserving populations.

Patients from equity-deserving populations, such as those who are from racialized communities, the 2SLGBTQI+ community, who are refugees or immigrants, and/or who have a disability, may experience a unique set of challenges accessing virtual models of care. The objective of this qualitative study was to describe the experiences of patients from equity-deserving communities and their family members who received care from a Virtual Emergency Department (ED) in Toronto, Canada. Forty-three participants (36 patients and 7 family caregivers) with different and intersecting identities who used the Virtual ED participated in the study. Semi-structured interviews were conducted to explore reasons for accessing the Virtual ED, barriers to access, and how the Virtual ED met their care needs and expectations, including ways their experience could have been improved. Thematic analysis was used to identify themes from the data. Patients from equity-deserving populations described negative past experiences with ED in-person care, which included recounts of discrimination or culturally insensitive care while waiting to see the ED physician or nurse. Conversely, participants found the Virtual ED to be a socially and culturally safe space since they could now by-pass the waiting room experience. However, virtual care could not replace in-person care for certain issues (e.g., physical exam), and there was a need for greater promotion of the service to specific communities that might benefit from having access to the Virtual ED. Targeted outreach to help raise awareness of the service to equity-deserving communities is an important future direction.

Learning From a Regional Approach: Integration to Scale, Spread, and Sustain Virtual Urgent Care.

While new offerings of virtual urgent care services from peer hospitals faltered after initial provincial pilot funding lapsed, our 3 regional academic health sciences centers decided to partner to enhance patient access, achieve efficiencies, and support long-term sustainability. Utilizing the Development Model for Integrated Care framework, we progressed through the 4 phases to ensure joint success and high-quality care: (1) initiative and design phase-individual parallel projects but with strong collaborations and broad stakeholder engagement; (2) experimental and execution phase-continuous quality improvement approach for governance, policies, and processes; (3) expansion and monitoring phase-weekly leadership touchpoints on key performance indicators; and (4) consolidation and transformation phase-sustainability through ongoing funding.

Training Residents for the Future: A Virtual Care Rotation for Emergency Medicine.

OBJECTIVES: Virtual care (VC) is increasingly becoming a part of emergency medicine (EM) physician workflows, yet no formal digital health curricula exist within Canadian EM training programs. The objective was to design and pilot a VC elective rotation for EM residents to help address this gap and better prepare them for future VC practice. METHODS: The current work describes the design and implementation of a 4-week VC elective rotation for EM residents. The rotation consisted of VC shifts, medical transport shifts, one-on-one discussions with various stakeholders, weekly thematic articles, and a final project deliverable. RESULTS: The rotation was well received by all stakeholders, and the quality of feedback and one-on-one teaching were highlighted as strengths. Future work will consider the optimal delivery timing of this type of curricula, whether all EM residents should receive basic training in VC, and how our current findings may be generalizable to other VC sites. CONCLUSION: A formal digital health curriculum for EM residents supports competency development for delivering VC as part of future EM practice.

The Implementation of a Virtual Emergency Department: Multimethods Study Guided by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) Framework.

BACKGROUND: While the COVID-19 pandemic dramatically increased virtual care uptake across many health settings, it remains significantly underused in urgent care. OBJECTIVE: This study evaluated the implementation of a pilot virtual emergency department (VED) at an Ontario hospital that connected patients to emergency physicians through a web-based portal. We sought to (1) assess the acceptability of the VED model, (2) evaluate whether the VED was implemented as intended, and (3) explore the impact on quality of care, access to care, and continuity of care. METHODS: This evaluation used a multimethods approach informed by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework. Data included semistructured interviews with patients and physicians as well as postvisit surveys from patients. Interviews were transcribed and analyzed using thematic analysis. Data from the surveys were described using summary statistics. RESULTS: From December 2020 to December 2021, the VED had a mean of 153 (SD 25) visits per month. Among them, 67% (n=677) were female, and 75% (n=758) had a family physician. Patients reported that the VED provided high-quality, timely access to care and praised the convenience, shorter appointments, and benefit of the calm, safe space afforded through virtual appointments. In instances where patients were directed to come into the emergency department (ED), physicians were able to provide a "warm handoff" to improve efficiency. This helped manage patient expectations, and the direct advice of the ED physician reassured them that the visit was warranted. There was broad initial uptake of VED shifts among ED physicians with 60% (n=22) completing shifts in the first 2 months and 42% (n=15) completing 1 or more shifts per month over the course of the pilot. There were no difficulties finding sufficient ED physicians for shifts. Most physicians enjoyed working in the VED, saw value for patients, and were motivated by patient satisfaction. However, some physicians were hesitant as they felt their expertise and skills as ED physicians were underused. The VED was implemented using an iterative staged approach with increased service capabilities over time, including access to ultrasounds, virtual follow-ups after a recent ED visit, and access to blood work, urine tests, and x-rays (at the hospital or a local community laboratory). Physicians recognized the value in supporting patients by advising on the need for an in-person visit, booking a diagnostic test, or referring them to a specialist. CONCLUSIONS: The VED had the support of physicians and facilitated care for low-acuity presentations with immediate benefits for patients. It has the potential to benefit the health care system by seeing patients through the web and guiding patients to in-person care only when necessary. Long-term sustainability requires a focus on understanding digital equity and enhanced access to rapid testing or investigations.

"It saved me from the emergency department": A qualitative study of patient experience of virtual urgent care in Ontario.

INTRODUCTION: In response to the COVID-19 pandemic, the Ontario Ministry of Health introduced a pilot program of 14 virtual urgent care (VUC) initiatives across the province to encourage physical distancing and provision of care by telephone and video-enabled visits. The implementation of the VUC pilot is currently being evaluated by an external academic team. The objective of this study was to understand patient experiences with VUC to determine barriers and facilitators to optimal virtual care as it rapidly expands during the current pandemic and beyond. METHOD: The qualitative component of the evaluation used one-on-one telephone interviews with patients, families, providers, and program administrators as the main method of data collection. Patient and family participants were invited to participate by the triage nurse after their VUC visit. Data analysis, using thematic analysis, occurred in conjunction with data collection to monitor emerging themes and areas for further exploration. RESULTS: Between April and October 2021, we completed 14 patient and/or family interviews from a representative cross-section of 6 pilot sites. Participants had a range of presenting complaints including infection, injury, medication side effects, and abdominal pain. The vast majority of participants were female (90%), and 70% were VUC patients themselves. Our analysis identified three key themes in the data which characterise patient and family member experience with VUC: a) emphasis on access to the ED; b) efficiency and quality of care; c) obtaining reassurance and next steps. CONCLUSION: Virtual care options are valued by patients and families; however, the nature of care needed by those accessing VUC and who can best provide that care needs to be evaluated to position it for sustainability. Understanding how virtual care performs from both a provider and patient perspective during the current crisis has implications for designing alternative care options beyond the COVID-19 pandemic.

Demographic characteristics, outcomes and experience of patients using virtual urgent care services from 14 emergency department led sites in Ontario.

INTRODUCTION: As part of the COVID-19 pandemic response, the Ontario Ministry of Health funded a virtual care pilot program intended to support emergency department (ED) diversion of patients with low acuity complaints and reduce the need for face-to-face contact. The objective was to describe the demographic characteristics, outcomes and experience of patients using the provincial pilot program. METHODS: This was a prospective cohort study of patients using virtual care services provided by 14 ED-led pilot sites from December 2020 to September 2021. Patients who completed a virtual visit were invited by email to complete a standardized, 25-item online survey, which included questions related to satisfaction and patient-reported outcome measures. RESULTS: There were 22,278 virtual visits. When patients were asked why they contacted virtual urgent care, of the 82.7% patients who had a primary care provider, 31.0% said they could not make a timely appointment with their family physician. Rash, fever, abdominal pain, and COVID-19 vaccine queries represented 30% of the presenting complaints. Of 19,613 patients with a known disposition, 12,910 (65.8%) were discharged home and 3,179 (16.2%) were referred to the ED. Of the 2,177 survey responses, 94% rated their overall experience as 8/10 or greater. More than 80% said they had answers to all the questions they had related to their health concern, believed they were able to manage the issue, had a plan they could follow, and knew what to do if the issue got worse or came back. CONCLUSIONS: Many presenting complaints were low acuity, and most patients had a primary care provider, but timely access was not available. Future work should focus on health equity to ensure virtual care is accessible to underserved populations. We question if virtual urgent care can be safely and more economically provided by non-emergency physicians.

RéSUMé: INTRODUCTION: Dans le cadre de la réponse à la pandémie de COVID-19, le ministère de la Santé de l'Ontario a financé un programme pilote de soins virtuels visant à soutenir la réorientation vers les services d'urgence des patients présentant des problèmes de faible acuité et à réduire le besoin de contact en personne. L'objectif était de décrire les caractéristiques démographiques, les résultats et l'expérience des patients utilisant le programme pilote provincial. MéTHODES: Il s'agissait d'une étude de cohorte prospective de patients utilisant des services de soins virtuels fournis par 14 sites pilotes dirigés par des services d'urgence, de décembre 2020 à septembre 2021. Les patients qui ont effectué une visite virtuelle ont été invités par courriel à répondre à une enquête en ligne standardisée de 25 questions, qui comprenait des questions relatives à la satisfaction et aux résultats rapportés par les patients. RéSULTATS: Il y a eu 22 278 visites virtuelles. Lorsqu'on a demandé aux patients pourquoi ils avaient contacté les soins urgents virtuels, sur les 82,7 % de patients qui avaient un prestataire de soins primaires, 31,0 % ont répondu qu'ils n'avaient pas pu obtenir un rendez-vous en temps voulu avec leur médecin de famille. Les éruptions cutanées, la fièvre, les douleurs abdominales et les interrogations sur le vaccin COVID-19 représentaient 30 % des plaintes présentées. Sur les 19 613 patients dont la disposition était connue, 12 910 (65,8 %) ont été renvoyés chez eux et 3 179 (16,2 %) ont été orientés vers les urgences. Sur les 2 177 réponses à l'enquête, 94 % ont attribué une note de 8/10 ou plus à leur expérience globale. Plus de 80 % d'entre eux ont déclaré avoir obtenu des réponses à toutes les questions qu'ils se posaient sur leur problème de santé, se croire capables de le gérer, avoir un plan qu'ils pouvaient suivre et savoir quoi faire si le problème s'aggravait ou revenait. CONCLUSIONS: De nombreuses plaintes présentées étaient de faible acuité, et la plupart des patients avaient un fournisseur de soins primaires, mais l'accès en temps opportun n'était pas disponible. Les travaux futurs devraient se concentrer sur l'équité en matière de santé pour s'assurer que les soins virtuels sont accessibles aux populations mal desservies et nous nous demandons si ces services peuvent être fournis en toute sécurité et de manière plus économique par des médecins non urgentistes.

Implementing a Virtual Emergency Department: Qualitative Study Using the Normalization Process Theory.

BACKGROUND: COVID-19 necessitated the rapid implementation and uptake of virtual health care; however, virtual care's potential role remains unclear in the urgent care setting. In December 2020, the first virtual emergency department (ED) in the Greater Toronto Area was piloted at Sunnybrook Health Sciences Centre by connecting patients to emergency physicians through an online portal. OBJECTIVE: This study aims to understand whether and how ED physicians were able to integrate a virtual ED alongside in-person operations. METHODS: We conducted semistructured interviews with ED physicians guided by the Normalization Process Theory (NPT). The NPT provides a framework to understand how individuals and teams navigate the process of embedding new models of care as part of normal practice. All physicians who had worked within the virtual ED model were invited to participate. Data were analyzed using a combination of inductive and deductive techniques informed by the NPT. RESULTS: A total of 14 physicians were interviewed. Participant experiences were categorized into 1 of 2 groups: 1 group moved to normalize the virtual ED in practice, while the other described barriers to routine adoption. These groups differed in their perception of the patient benefits as well as the perceived role in the virtual ED. The group that normalized the virtual ED model saw value for patients (coherence) and was motivated by patient satisfaction witnessed (reflexive monitoring) at the end of the virtual appointment. By contrast, the other group did not find virtual ED work reflective of the perceived role of urgent care (cognitive participation) and felt their skills as ED physicians were underutilized. The limited ability to examine patients and a sense that patient issues were not fully resolved at the end of the virtual appointment caused frustration among the second group. CONCLUSIONS: As further digital integration within the health care system occurs, it will be essential to support the evolution of staff skill sets to ensure physicians are satisfied with the care they are providing to their patients, while also ensuring the technology and process are efficient.

Designs, facilitators, barriers, and lessons learned during the implementation of emergency department led virtual urgent care programs in Ontario, Canada.

Introduction: Virtual patient care has seen incredible growth since the beginning of the COVID-19 pandemic. To provide greater access to safe and timely urgent care, in the fall of 2020, the Ministry of Health introduced a pilot program of 14 virtual urgent care (VUC) initiatives across the province of Ontario. The objective of this paper was to describe the overall design, facilitators, barriers, and lessons learned during the implementation of seven emergency department (ED) led VUC pilot programs in Ontario, Canada. Methods: We assembled an expert panel of 13 emergency medicine physicians and researchers with experience leading and implementing local VUC programs. Each VUC program lead was asked to describe their local pilot program, share common facilitators and barriers to adoption of VUC services, and summarize lessons learned for future VUC design and development. Results: Models of care interventions varied across VUC pilot programs related to triage, staffing, technology, and physician remuneration. Common facilitators included local champions to guide program delivery, provincial funding support, and multi-modal marketing and promotions. Common barriers included behaviour change strategies to support adoption of a new service, access to high-quality information technology to support new workflow models that consider privacy, risk, and legal perspectives, and standardized data collection which underpin overall objective impact assessments. Conclusions: These pilot programs were rapidly implemented to support safe access to care and ED diversion of patients with low acuity issues during the COVID-19 pandemic. The heterogeneity of program implementation respects local autonomy yet may present challenges for sustainability efforts and future funding considerations.

Assessing the congruence between perceived connectivity and network centrality measures specific to pandemic influenza preparedness in Alberta.

BACKGROUND: Recent research has suggested that perceived organizational connectivity may serve as an important measure of public health preparedness. Presumably, organizations with higher perceived connectivity also have a greater number of actual organizational ties. Using network analysis, we evaluate this presumption by assessing the correlation between perceived organizational connectivity and reported inter-organizational connections. METHODS: During late 2007-early 2008, representatives from organizations involved in the delivery of public health systems in Alberta were asked to complete an online questionnaire on public health preparedness. Organizational jurisdictional information was collected. Items from Dorn and colleagues connectivity scale (2007) were used to measure perceived organizational connectivity. Inter-organizational network data on formal connections in the area of pandemic influenza preparedness were collected using a roster approach. These data were imported into UCINET to calculate in- and out-degree centrality scores for each organization. One-way ANOVA tests assessed if perceived connectivity and in- and out-degree centrality varied among jurisdictions. Pearson correlation coefficients were used to assess the correlation of perceived connectivity and in- and out-degree centrality. RESULTS: Significant mean differences among jurisdictions were observed for in-degree (F(3,116) = 26.60, p < 0.001) and between provincial and lower jurisdictions for out-degree centrality (F(3,116) = 5.24, p < 0.01). Higher jurisdictions had higher average centrality. Perceived organizational connectivity was correlated with out-degree (r(123) = 0.22, p < 0.05) but not in-degree centrality (r(123) = -0.07, p > 0.05). CONCLUSIONS: The results suggest in terms of pandemic preparedness that perceived connectivity may serve as a partial proxy measure of formal out-degree network connectivity.

Automated mortality surveillance in south-eastern Ontario for pandemic influenza preparedness.

BACKGROUND: The recent Canadian experience with pandemic H1N1 (pH1N1) influenza in 2009 highlighted the need for enhanced surveillance at local and regional levels to support evidence-based decision making by physicians and public health. We describe the rationale, methodology, and provide preliminary findings from the implementation of an automated Mortality Surveillance System (MSS) in the Kingston, Frontenac and Lennox & Addington (KFL&A) health unit. METHODS: The MSS utilized an automated web-based framework with secure data transfer. A data sharing agreement between the local Medical Officer of Health and the City of Kingston facilitated weekly updates of mortality data. Deaths due to influenza were classified using keywords in the cause of death and a phonetic algorithm to capture alternate spellings. Anomaly detection was modeled on the modified cumulative sum algorithm implemented in the Early Aberration Reporting System. RESULTS: Retrospective analysis of municipal mortality data over a 10-year period established baseline mortality rates in the region. MSS data monitored during the pH1N1 influenza season showed no significant impact on the burden or timing of mortality in the KFL&A health unit. CONCLUSION: Municipal data enabled surveillance of mortality in the KFL&A region with weekly updates. Other municipalities may participate in this surveillance project using the Kingston model without significant ongoing investment. Efforts to improve data quality at the physician and transcription level are ongoing. Integration of mortality data and other real-time data streams into an integrated electronic public health dashboard could provide decision-makers with timely information during public health emergencies.

Residents' perspective of quality improvement and patient safety education in Canadian emergency medicine residency programs.

OBJECTIVES: Quality improvement and patient safety (QIPS) competencies are increasingly important in emergency medicine (EM) and are now included in the CanMEDS framework. We conducted a survey aimed at determining the Canadian EM residents' perspectives on the level of QIPS education and support available to them. METHODS: An electronic survey was distributed to all Canadian EM residents from the Royal College and Family Medicine training streams. The survey consisted of multiple-choice, Likert, and free-text entry questions aimed at understanding familiarity with QIPS, local opportunities for QIPS projects and mentorship, and the desire for further QIPS education and involvement. RESULTS: Of 535 EM residents, 189 (35.3%) completed the survey, representing all 17 medical schools; 77.2% of respondents were from the Royal College stream; 17.5% of respondents reported that QIPS methodologies were formally taught in their residency program; 54.7% of respondents reported being "somewhat" or "very" familiar with QIPS; 47.2% and 51.5% of respondents reported either "not knowing" or "not having readily available" opportunities for QIPS projects and QIPS mentorship, respectively; 66.9% of respondents indicated a desire for increased QIPS teaching; and 70.4% were interested in becoming involved with QIPS training and initiatives. CONCLUSIONS: Many Canadian EM residents perceive a lack of QIPS educational opportunities and support in their local setting. They are interested in receiving more QIPS education, as well as project and mentorship opportunities. Supporting residents with a robust QIPS educational and mentorship framework may build a cohort of providers who can enhance the local delivery of care.

Faculty development in the age of competency-based medical education: A needs assessment of Canadian emergency medicine faculty and senior trainees.

OBJECTIVES: The Royal College of Physicians and Surgeons of Canada (RCPSC) emergency medicine (EM) programs transitioned to the Competence by Design training framework in July 2018. Prior to this transition, a nation-wide survey was conducted to gain a better understanding of EM faculty and senior resident attitudes towards the implementation of this new program of assessment. METHODS: A multi-site, cross-sectional needs assessment survey was conducted. We aimed to document perceptions about competency-based medical education, attitudes towards implementation, perceived/prompted/unperceived faculty development needs. EM faculty and senior residents were nominated by program directors across RCPSC EM programs. Simple descriptive statistics were used to analyse the data. RESULTS: Between February and April 2018, 47 participants completed the survey (58.8% response rate). Most respondents (89.4%) thought learners should receive feedback during every shift; 55.3% felt that they provided adequate feedback. Many respondents (78.7%) felt that the ED would allow for direct observation, and most (91.5%) participants were confident that they could incorporate workplace-based assessments (WBAs). Although a fair number of respondents (44.7%) felt that Competence by Design would not impact patient care, some (17.0%) were worried that it may negatively impact it. Perceived faculty development priorities included feedback delivery, completing WBAs, and resident promotion decisions. CONCLUSIONS: RCPSC EM faculty have positive attitudes towards competency-based medical education-relevant concepts such as feedback and opportunities for direct observation via WBAs. Perceived threats to Competence by Design implementation included concerns that patient care and trainee education might be negatively impacted. Faculty development should concentrate on further developing supervisors' teaching skills, focusing on feedback using WBAs.

INTRODUCTION: Les programmes de médecine d'urgence (MU) du Collège royal des médecins et chirurgiens du Canada sont passés, en juillet 2018, à un nouveau modèle de formation appelé Compétence par conception. Avant ce passage, une enquête avait été menée à l'échelle nationale pour dégager une meilleure compréhension des attitudes du personnel enseignant et des résidents séniors en MU à l'égard de la mise en œuvre du nouveau programme d'évaluation. MÉTHODE: Il s'agit d'une enquête transversale et multicentrique sur l'évaluation des besoins. Elle visait à recueillir des renseignements sur les perceptions des personnes concernées sur la formation médicale axée sur les compétences, sur leurs attitudes à l'égard de la mise en application du programme ainsi que sur les besoins perçus, suscités ou inaperçus du personnel en matière de perfectionnement. Les membres du personnel enseignant en MU et les résidents séniors ont été désignés par les directeurs de tous les programmes de MU du Collège royal. Les données ont été analysées à l'aide de simples statistiques descriptives. RÉSULTATS: Au total, 47 participants ont rempli le questionnaire d'enquête (taux de réponse : 58,8%) entre février et avril 2018. La plupart des répondants (89,4%) ont indiqué que les apprenants devraient recevoir de la rétroaction à tous les postes de travail, et 55,3% avaient l'impression de donner une rétroaction adéquate. Bon nombre de répondants (78,7%) étaient également d'avis que les services des urgences se prêtaient bien à l'observation directe, et la plupart des participants (91,5%) avaient bon espoir d'intégrer les évaluations en milieu de travail (EMT). Par ailleurs, si un assez bon nombre de répondants (44,7%) croyaient que la formation axée sur la Compétence par conception n'aurait aucune incidence sur les soins aux patients, d'autres (17,0%) s'en inquiétaient. Enfin, les priorités perçues en matière de perfectionnement du personnel comprenaient la communication des rétroactions, la réalisation des EMT et les décisions relatives à la promotion des résidents. CONCLUSION: Le personnel enseignant en MU du Collège royal a des attitudes favorables à l'égard de certains concepts liés à la formation médicale axée sur les compétences, tels que les rétroactions et les possibilités d'observation directe au moyen des EMT. Par contre, la mise en œuvre du programme de Compétence par conception suscite également des craintes, comme une incidence défavorable sur les soins aux patients et la formation des stagiaires. Le perfectionnement du personnel enseignant devrait donc porter davantage sur les aptitudes à enseigner des superviseurs, notamment sur la communication des rétroactions à l'aide des EMT.