Quality improvement strategies for diabetes care: Effects on outcomes for adults living with diabetes.

BACKGROUND: There is a large body of evidence evaluating quality improvement (QI) programmes to improve care for adults living with diabetes. These programmes are often comprised of multiple QI strategies, which may be implemented in various combinations. Decision-makers planning to implement or evaluate a new QI programme, or both, need reliable evidence on the relative effectiveness of different QI strategies (individually and in combination) for different patient populations. OBJECTIVES: To update existing systematic reviews of diabetes QI programmes and apply novel meta-analytical techniques to estimate the effectiveness of QI strategies (individually and in combination) on diabetes quality of care. SEARCH METHODS: We searched databases (CENTRAL, MEDLINE, Embase and CINAHL) and trials registers (ClinicalTrials.gov and WHO ICTRP) to 4 June 2019. We conducted a top-up search to 23 September 2021; we screened these search results and 42 studies meeting our eligibility criteria are available in the awaiting classification section. SELECTION CRITERIA: We included randomised trials that assessed a QI programme to improve care in outpatient settings for people living with diabetes. QI programmes needed to evaluate at least one system- or provider-targeted QI strategy alone or in combination with a patient-targeted strategy. - System-targeted: case management (CM); team changes (TC); electronic patient registry (EPR); facilitated relay of clinical information (FR); continuous quality improvement (CQI). - Provider-targeted: audit and feedback (AF); clinician education (CE); clinician reminders (CR); financial incentives (FI). - Patient-targeted: patient education (PE); promotion of self-management (PSM); patient reminders (PR). Patient-targeted QI strategies needed to occur with a minimum of one provider or system-targeted strategy. DATA COLLECTION AND ANALYSIS: We dual-screened search results and abstracted data on study design, study population and QI strategies. We assessed the impact of the programmes on 13 measures of diabetes care, including: glycaemic control (e.g. mean glycated haemoglobin (HbA1c)); cardiovascular risk factor management (e.g. mean systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), proportion of people living with diabetes that quit smoking or receiving cardiovascular medications); and screening/prevention of microvascular complications (e.g. proportion of patients receiving retinopathy or foot screening); and harms (e.g. proportion of patients experiencing adverse hypoglycaemia or hyperglycaemia). We modelled the association of each QI strategy with outcomes using a series of hierarchical multivariable meta-regression models in a Bayesian framework. The previous version of this review identified that different strategies were more or less effective depending on baseline levels of outcomes. To explore this further, we extended the main additive model for continuous outcomes (HbA1c, SBP and LDL-C) to include an interaction term between each strategy and average baseline risk for each study (baseline thresholds were based on a data-driven approach; we used the median of all baseline values reported in the trials). Based on model diagnostics, the baseline interaction models for HbA1c, SBP and LDL-C performed better than the main model and are therefore presented as the primary analyses for these outcomes. Based on the model results, we qualitatively ordered each QI strategy within three tiers (Top, Middle, Bottom) based on its magnitude of effect relative to the other QI strategies, where 'Top' indicates that the QI strategy was likely one of the most effective strategies for that specific outcome. Secondary analyses explored the sensitivity of results to choices in model specification and priors.  Additional information about the methods and results of the review are available as Appendices in an online repository. This review will be maintained as a living systematic review; we will update our syntheses as more data become available. MAIN RESULTS: We identified 553 trials (428 patient-randomised and 125 cluster-randomised trials), including a total of 412,161 participants. Of the included studies, 66% involved people living with type 2 diabetes only. Participants were 50% female and the median age of participants was 58.4 years. The mean duration of follow-up was 12.5 months. HbA1c was the commonest reported outcome; screening outcomes and outcomes related to cardiovascular medications, smoking and harms were reported infrequently. The most frequently evaluated QI strategies across all study arms were PE, PSM and CM, while the least frequently evaluated QI strategies included AF, FI and CQI. Our confidence in the evidence is limited due to a lack of information on how studies were conducted.  Four QI strategies (CM, TC, PE, PSM) were consistently identified as 'Top' across the majority of outcomes. All QI strategies were ranked as 'Top' for at least one key outcome. The majority of effects of individual QI strategies were modest, but when used in combination could result in meaningful population-level improvements across the majority of outcomes. The median number of QI strategies in multicomponent QI programmes was three.  Combinations of the three most effective QI strategies were estimated to lead to the below effects:  - PR + PSM + CE: decrease in HbA1c by 0.41% (credibility interval (CrI) -0.61 to -0.22) when baseline HbA1c < 8.3%; - CM + PE + EPR: decrease in HbA1c by 0.62% (CrI -0.84 to -0.39) when baseline HbA1c > 8.3%;  - PE + TC + PSM: reduction in SBP by 2.14 mmHg (CrI -3.80 to -0.52) when baseline SBP < 136 mmHg; - CM + TC + PSM: reduction in SBP by 4.39 mmHg (CrI -6.20 to -2.56) when baseline SBP > 136 mmHg;  - TC + PE + CM: LDL-C lowering of 5.73 mg/dL (CrI -7.93 to -3.61) when baseline LDL < 107 mg/dL; - TC + CM + CR: LDL-C lowering by 5.52 mg/dL (CrI -9.24 to -1.89) when baseline LDL > 107 mg/dL. Assuming a baseline screening rate of 50%, the three most effective QI strategies were estimated to lead to an absolute improvement of 33% in retinopathy screening (PE + PR + TC) and 38% absolute increase in foot screening (PE + TC + Other). AUTHORS' CONCLUSIONS: There is a significant body of evidence about QI programmes to improve the management of diabetes. Multicomponent QI programmes for diabetes care (comprised of effective QI strategies) may achieve meaningful population-level improvements across the majority of outcomes. For health system decision-makers, the evidence summarised in this review can be used to identify strategies to include in QI programmes. For researchers, this synthesis identifies higher-priority QI strategies to examine in further research regarding how to optimise their evaluation and effects. We will maintain this as a living systematic review.

Interprofessional and multiprofessional approaches in quality improvement education.

The imperative for all healthcare professionals to partake in quality improvement (QI) has resulted in the development of QI education programs with participants from different professional backgrounds. However, there is limited empirical and theoretical examination as to why, when and how interprofessional and multiprofessional education occurs in QI and the outcomes of these approaches. This paper reports on a qualitative collective case study of interprofessional and multiprofessional education in three longitudinal QI education programs. We conducted 58 interviews with learners, QI project coaches, program directors and institutional leads and 135 h of observations of in-class education sessions, and collected relevant documents such as course syllabi and handouts. We used an interpretive thematic analysis using a conventional and directed content analysis approach. In the directed content approach, we used sociology of professions theory with particular attention to professional socialization, hierarchies and boundaries in QI, to understand the ways in which individuals' professional backgrounds informed the planning and experiences of the QI education programs. Findings demonstrated that both interprofessional and multiprofessional education approaches were being used to achieve different education objectives. While each approach demonstrated positive learning and practice outcomes, tensions related to the different ways in which professional groups are engaging in QI, power dynamics between professional groups, and disconnects between curricula and practice existed. Further conceptual clarity is essential for a more informed discussion about interprofessional and multiprofessional education approaches in QI and explicit attention is needed to professional processes and tensions, to optimize the impact of education on practice.

Building Bridges to Integrate Care (BRIDGES): Incubating Health Service Innovation across the Continuum of Care for Patients with Multiple Chronic Conditions.

Integrating care for people with complex needs is challenging. Indeed, evidence of solutions is mixed, and therefore, well-designed, shared evaluation approaches are needed to create cumulative learning. The Toronto-based Building Bridges to Integrate Care (BRIDGES) collaborative provided resources to refine and test nine new models linking primary, hospital and community care. It used mixed methods, a cross-project meta-evaluation and shared outcome measures. Given the range of skills required to develop effective interventions, a novel incubator was used to test and spread opportunities for system integration that included operational expertise and support for evaluation and process improvement.

Reducing antimicrobial therapy for asymptomatic bacteriuria among noncatheterized inpatients: a proof-of-concept study.

This proof-of-concept study demonstrates that no longer routinely reporting urine culture results from noncatheterized medical and surgical inpatients can greatly reduce unnecessary antimicrobial therapy for asymptomatic bacteriuria without significant additional laboratory workload. Larger studies are needed to confirm the generalizability, safety, and sustainability of this model of care.

Impact of stated barriers on proposed warfarin prescription for atrial fibrillation: a survey of Canadian physicians.

BACKGROUND: Atrial fibrillation (AF) is a common cardiac arrhythmia, and leading cause of ischemic stroke. Despite proven effectiveness, warfarin remains an under-used treatment in atrial fibrillation patients. We sought to study, across three physician specialties, a range of factors that have been argued to have a disproportionate effect on treatment decisions. METHODS: Cross-sectional survey of Canadian Family Doctors (FD: n = 500), Geriatricians (G: n = 149), and Internal Medicine specialists (IMS: n = 500). Of these, 1032 physicians were contactable, and 335 completed and usable responses were received. Survey questions and clinical vignettes asked about the frequency with which they see patients with atrial fibrillation, treatment practices, and barriers to the prescription of anticoagulants. RESULTS: Stated prescribing practices did not significantly differ between physician groups. Falls risk, bleeding risk and poor patient adherence were all highly cited barriers to prescribing warfarin. Fewer geriatricians indicated that history of patient falls would be a reason for not treating with warfarin (G: 47%; FD: 71%; IMS: 72%), and significantly fewer changed reported practice in the presence of falls risk (χ (2) (6) = 45.446, p < 0.01). Experience of a patient having a stroke whilst not on warfarin had a significant impact on vignette decisions; physicians who had had patients who experienced a stroke were more likely to prescribe warfarin (χ (2) (3) =10.7, p = 0.013). CONCLUSIONS: Barriers to treatment of atrial fibrillation with warfarin affect physician specialties to different extents. Prior experience of a patient suffering a stroke when not prescribed warfarin is positively associated with intention to prescribe warfarin, even in the presence of falls risk.

Medication reconciliation during transitions of care as a patient safety strategy: a systematic review.

Medication reconciliation identifies and resolves unintentional discrepancies between patients' medication lists across transitions in care. The purpose of this review is to summarize evidence about the effectiveness of hospital-based medication reconciliation interventions. Searches encompassed MEDLINE through November 2012 and EMBASE and the Cochrane Central Register of Controlled Trials through July 2012. Eligible studies evaluated the effects of hospital-based medication reconciliation on unintentional discrepancies with nontrivial risks for harm to patients or 30-day postdischarge emergency department visits and readmission. Two reviewers evaluated study eligibility, abstracted data, and assessed study quality. Eighteen studies evaluating 20 interventions met the selection criteria. Pharmacists performed medication reconciliation in 17 of the 20 interventions. Most unintentional discrepancies identified had no clinical significance. Medication reconciliation alone probably does not reduce postdischarge hospital utilization but may do so when bundled with interventions aimed at improving care transitions.

A Qualitative Analysis of Physician Perspectives on Missed and Delayed Outpatient Diagnosis: The Focus on System-Related Factors.

BACKGROUND: Delayed and missed diagnoses lead to significant patient harm. Because physician actions are fundamental to the outpatient diagnostic process, a study was conducted to explore physician perspectives on diagnosis. METHODS: As part of a quality improvement initiative, an integrated health system conducted six physician focus groups in 2004 and 2005. The focus groups included questions about the process of diagnosis, specific factors contributing to missed diagnosis, use of guidelines, atypical vs. typical presentations of disease, diagnostic tools, and follow-up, all with regard to delays in the diagnostic process. The interviews were analyzed (1) deductively, with application of the Systems Engineering Initiative for Patient Safety (SEIPS) model, which addresses systems design, quality management, job design, and technology implementations that affect safety-related patient and organizational and/or staff outcomes, and (2) inductively, with identification of novel themes using content analysis. RESULTS: A total of 25 physicians participated in the six focus groups, which yielded 12 hours of discussion. Providers identified multiple barriers to timely and accurate diagnosis, including organizational culture, information availability, and communication factors. CONCLUSIONS: Multiple themes relating to each of the participants in the diagnostic process-health system, provider, and patient-emerged. Concerns about health system structure and providers' interactions with one another and with patients far exceeded discussion of the cognitive factors that might affect the diagnostic process. The results suggest that, at least in physicians' views, improving the diagnostic process requires attention to the organization of the health system in addition to the cognitive aspects of diagnosis.

Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives.

Machine learning (ML) solutions are increasingly entering healthcare. They are complex, sociotechnical systems that include data inputs, ML models, technical infrastructure and human interactions. They have promise for improving care across a wide range of clinical applications but if poorly implemented, they may disrupt clinical workflows, exacerbate inequities in care and harm patients. Many aspects of ML solutions are similar to other digital technologies, which have well-established approaches to implementation. However, ML applications present distinct implementation challenges, given that their predictions are often complex and difficult to understand, they can be influenced by biases in the data sets used to develop them, and their impacts on human behaviour are poorly understood. This manuscript summarises the current state of knowledge about implementing ML solutions in clinical care and offers practical guidance for implementation. We propose three overarching questions for potential users to consider when deploying ML solutions in clinical care: (1) Is a clinical or operational problem likely to be addressed by an ML solution? (2) How can an ML solution be evaluated to determine its readiness for deployment? (3) How can an ML solution be deployed and maintained optimally? The Quality Improvement community has an essential role to play in ensuring that ML solutions are translated into clinical practice safely, effectively, and ethically.

Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis.

BACKGROUND: The effectiveness of quality improvement (QI) strategies on diabetes care remains unclear. We aimed to assess the effects of QI strategies on glycated haemoglobin (HbA(1c)), vascular risk management, microvascular complication monitoring, and smoking cessation in patients with diabetes. METHODS: We identified studies through Medline, the Cochrane Effective Practice and Organisation of Care database (from inception to July 2010), and references of included randomised clinical trials. We included trials assessing 11 predefined QI strategies or financial incentives targeting health systems, health-care professionals, or patients to improve management of adult outpatients with diabetes. Two reviewers independently abstracted data and appraised risk of bias. FINDINGS: We reviewed 48 cluster randomised controlled trials, including 2538 clusters and 84,865 patients, and 94 patient randomised controlled trials, including 38,664 patients. In random effects meta-analysis, the QI strategies reduced HbA(1c) by a mean difference of 0·37% (95% CI 0·28-0·45; 120 trials), LDL cholesterol by 0·10 mmol/L (0·05-0.14; 47 trials), systolic blood pressure by 3·13 mm Hg (2·19-4·06, 65 trials), and diastolic blood pressure by 1·55 mm Hg (0·95-2·15, 61 trials) versus usual care. We noted larger effects when baseline concentrations were greater than 8·0% for HbA(1c), 2·59 mmol/L for LDL cholesterol, and 80 mm Hg for diastolic and 140 mm Hg for systolic blood pressure. The effectiveness of QI strategies varied depending on baseline HbA(1c) control. QI strategies increased the likelihood that patients received aspirin (11 trials; relative risk [RR] 1·33, 95% CI 1·21-1·45), antihypertensive drugs (ten trials; RR 1·17, 1·01-1·37), and screening for retinopathy (23 trials; RR 1·22, 1·13-1·32), renal function (14 trials; RR 128, 1·13-1·44), and foot abnormalities (22 trials; RR 1·27, 1·16-1·39). However, statin use (ten trials; RR 1·12, 0·99-1·28), hypertension control (18 trials; RR 1·01, 0·96-1·07), and smoking cessation (13 trials; RR 1·13, 0·99-1·29) were not significantly increased. INTERPRETATION: Many trials of QI strategies showed improvements in diabetes care. Interventions targeting the system of chronic disease management along with patient-mediated QI strategies should be an important component of interventions aimed at improving diabetes management. Interventions solely targeting health-care professionals seem to be beneficial only if baseline HbA(1c) control is poor. FUNDING: Ontario Ministry of Health and Long-term Care and the Alberta Heritage Foundation for Medical Research (now Alberta Innovates--Health Solutions).

Continuing medical education and quality improvement: a match made in heaven?

New models of continuing medical education (CME) seek not only to impart knowledge but to change physicians' behavior and even play a role in facilitating organizational improvement. These CME models thus share some of the same basic goals as the field of quality improvement (QI), namely behavioral change and systems redesign to improve patient outcomes. This article provides some practical ideas about how CME providers and QI experts may beneficially integrate these 2 fields. It outlines several models for harnessing the existing engagement in traditional CME to achieve the goal of equipping practitioners with knowledge and skills related to QI, while also addressing the widely recognized problems with traditional CME. The authors touch on possible incentives to make such integrated models of CME and QI attractive to practitioners.

Potentially avoidable admissions to general internal medicine at an academic teaching hospital: an observational study.

BACKGROUND: Identifying potentially avoidable admissions to Canadian hospitals is an important health system goal. With general internal medicine (GIM) accounting for 40% of hospital admissions, we sought to develop a method to identify potentially avoidable admissions and characterize patient, provider and health system factors. METHODS: We conducted an observational study of GIM admissions at our institution from August 2019 to February 2020. We defined potentially avoidable admissions as admissions that could be managed in an appropriate and safe manner in the emergency department or ambulatory setting and asked staff physicians to screen admissions daily and flag candidates as potentially avoidable admissions. For each candidate, we prepared a case review and debriefed with members of the admitting team. We then reviewed each candidate with our research team, assigned an avoidability score (1 [low] to 4 [high]) and identified contributing factors for those with scores of 3 or more. RESULTS: We screened 601 total admissions and staff physicians flagged 117 (19.5%) of these as candidate potential avoidable admissions. Consensus review identified 67 candidates as potentially avoidable admissions (11.1%, 95% confidence interval 8.8%-13.9%); these patients were younger (mean age 65 yr v. 72 yr), had fewer comorbidities (Canadian Institute for Health Information Case Mix Group+ 0.42 v. 1.14), had lower resource-intensity weighting scores (0.72 v. 1.50) and shorter hospital lengths of stay (29 h v. 105 h) (p < 0.01). Common factors included diagnostic and therapeutic uncertainty, perceived need for short-term monitoring, government directive of a 4-hour limit for admission decision-making and subspecialist request to admit. INTERPRETATION: Our prospective method of screening, flagging and case review showed that 1 in 9 GIM admissions were potentially avoidable. Other institutions could consider adapting this methodology to ascertain their rate of potentially avoidable admissions and to understand contributing factors to inform improvement endeavours.

Temporal Clustering of Critical Illness Events on Medical Wards.

Importance: Recognizing and preventing patient deterioration is important for hospital safety. Objective: To investigate whether critical illness events (in-hospital death or intensive care unit [ICU] transfer) are associated with greater risk of subsequent critical illness events for other patients on the same medical ward. Design, Setting, and Participants: Retrospective cohort study in 5 hospitals in Toronto, Canada, including 118 529 hospitalizations. Patients were admitted to general internal medicine wards between April 1, 2010, and October 31, 2017. Data were analyzed between January 1, 2020, and April 10, 2023. Exposures: Critical illness events (in-hospital death or ICU transfer). Main Outcomes and Measures: The primary outcome was the composite of in-hospital death or ICU transfer. The association between critical illness events on the same ward across 6-hour intervals was studied using discrete-time survival analysis, adjusting for patient and situational factors. The association between critical illness events on different comparable wards in the same hospital was measured as a negative control. Results: The cohort included 118 529 hospitalizations (median age, 72 years [IQR, 56-83 years]; 50.7% male). Death or ICU transfer occurred in 8785 hospitalizations (7.4%). Patients were more likely to experience the primary outcome after exposure to 1 prior event (adjusted odds ratio [AOR], 1.39; 95% CI, 1.30-1.48) and more than 1 prior event (AOR, 1.49; 95% CI, 1.33-1.68) in the prior 6-hour interval compared with no exposure. The exposure was associated with increased odds of subsequent ICU transfer (1 event: AOR, 1.67; 95% CI, 1.54-1.81; >1 event: AOR, 2.05; 95% CI, 1.79-2.36) but not death alone (1 event: AOR, 1.08; 95% CI, 0.97-1.19; >1 event: AOR, 0.88; 95% CI, 0.71-1.09). There was no significant association between critical illness events on different wards within the same hospital. Conclusions and Relevance: Findings of this cohort study suggest that patients are more likely to be transferred to the ICU in the hours after another patient's critical illness event on the same ward. This phenomenon could have several explanations, including increased recognition of critical illness and preemptive ICU transfers, resource diversion to the first event, or fluctuations in ward or ICU capacity. Patient safety may be improved by better understanding the clustering of ICU transfers on medical wards.

Quality improvement in medical education: current state and future directions.

CONTEXT: During the last decade, there has been a drive to improve the quality of patient care and prevent the occurrence of avoidable errors. This review describes current efforts to teach or engage trainees in patient safety and quality improvement (QI), summarises progress to date, as well as successes and challenges, and lists our recommendations for the next steps that will shape the future of patient safety and QI in medical education. CURRENT STATUS: Trainees encounter patient safety and QI through three main groups of activity. First are formal curricula that teach concepts or methods intended to facilitate trainees' participation in QI activities. These curricula increase learner knowledge and may improve clinical processes, but demonstrate limited capacity to modify learner behaviours. Second are educational activities that impart specific skills related to safety or quality which are considered to represent core doctor competencies (e.g. effective patient handover). These are frequently taught effectively, but without emphasis on the general safety or quality principles that inform the relevant skills. Third are real-life QI initiatives that involve trainees as active or passive participants. These innovative approaches expose trainees to safety and quality by integrating QI activities into trainees' day-to-day work. However, this integration can be challenging and can sometimes result in tension with broader educational goals. FUTURE DIRECTIONS: To prepare the next generation of doctors to make meaningful contributions to the quality mission, we propose the following call to action. Firstly, a major effort to build faculty capacity, especially among teachers of QI, should be instigated. Secondly, accreditation standards and assessment methods, both during training and at end-of-training certification examinations, should explicitly target these competencies. Finally, and perhaps most importantly, we must refocus our attention at all levels of training and instil fundamental, collaborative, open-minded behaviours so that future clinicians are primed to promote a culture of safer, higher-quality care.

Computerised provider order entry and residency education in an academic medical centre.

CONTEXT: Many academic medical centres (AMCs) have introduced institutional policies, changed processes of care and implemented new technologies to improve health care quality. The impact of such changes on medical education has received little attention. We examine the impact of computerised provider order entry (CPOE) on the educational experiences of medical trainees who work and train in AMCs. METHODS: We conducted semi-structured interviews of postgraduate trainees and attending physicians in internal medicine at five AMCs (two with CPOE, three without CPOE). Trainees routinely rotate from CPOE to non-CPOE AMCs, whereas some attending physicians work at both types of AMC and are therefore well positioned to reflect on differences between CPOE and non-CPOE learning environments. Data collection and analysis used grounded theory methods. We sampled purposively until we achieved theoretical saturation. RESULTS: Our study included 11 residents and six attending physicians. Computerised provider order entry had both positive and negative impacts on five aspects of postgraduate training: (i) learning (better for medication interactions and availability of learning resources; worse for learning medication doses); (ii) teaching (more medication information available to enhance case discussions; fewer face-to-face teaching opportunities); (iii) feedback (improved ability to observe medication ordering behaviours to inform feedback; less provision of direct feedback); (iv) clinical supervision (facilitates efficient and safe supervision from a distance; may impede trainee independence), and (v) trainee assessment (increased opportunity to assess clinical decision-making and organisational skills). CONCLUSIONS: We identify five key educational themes that are positively and negatively impacted by CPOE. These themes form a conceptual framework that could be applied to define the educational impact of other health care quality and patient safety practices. This will help educators to identify educational opportunities and protect the safety of the training experience of residents in AMCs.

Patient safety in emergency medical services: a systematic review of the literature.

BACKGROUND: Preventable harm from medical care has been extensively documented in the inpatient setting. Emergency medical services (EMS) providers care for patients in dynamic and challenging environments; prehospital emergency care is a field that represents an area of high risk for errors and harm, but has received relatively little attention in the patient safety literature. OBJECTIVE: To identify the threats to patient safety unique to the EMS environment and interventions that mitigate those threats, we completed a systematic review of the literature. METHODS: We searched MEDLINE, EMBASE, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) for combinations of key EMS and patient safety terms composed by a pan-canadian expert panel using a year limit of 1999 to 2011. We excluded commentaries, opinions, letters, abstracts, and non-english publications. Two investigators performed an independent hierarchical screening of titles, abstracts, and full-text articles blinded to source. We used the kappa statistic to examine interrater agreement. Any differences were resolved by consensus. RESULTS: We retrieved 5,959 titles, and 88 publications met the inclusion criteria and were categorized into seven themes: adverse events and medication errors (22 articles), clinical judgment (13), communication (6), ground vehicle safety (9), aircraft safety (6), interfacility transport (16), and intubation (16). Two articles were randomized controlled trials; the remainder were systematic reviews, prospective observational studies, retrospective database/chart reviews, qualitative interviews, or surveys. The kappa statistics for titles, abstracts, and full-text articles were 0.65, 0.79, and 0.87, respectively, for the first search and 0.60, 0.74, and 0.85 for the second. CONCLUSIONS: We found a paucity of scientific literature exploring patient safety in EMS. Research is needed to improve our understanding of problem magnitude and threats to patient safety and to guide interventions.

Order sets in health care: a systematic review of their effects.

OBJECTIVES: Order sets are widely used in hospitals to enter diagnosis and treatment orders. To determine the effectiveness of order sets in improving guideline adherence, treatment outcomes, processes of care, efficiency, and cost, we conducted a systematic review of the literature. METHODS: A comprehensive literature search was performed in various databases for studies published between January 1, 1990, and April 18, 2009. A total of eighteen studies met inclusion criteria. No randomized controlled trials were found. RESULTS: Outcomes of the included studies were summarized qualitatively due to variations in study population, intervention type, and outcome measures. There were no important inconsistencies between the results reported by studies involving different types of order sets. While the studies generally suggested positive outcomes, they were typically of low quality, with simple before-after designs and other methodological limitations. CONCLUSIONS: The benefits of order sets remain eminently plausible, but given the paucity of high quality evidence, further investigations to formally evaluate the effectiveness of order sets would be highly valuable.