A modified Delphi study to identify the features of high quality measurement plans for healthcare improvement projects.

BACKGROUND: The design and execution of measurement in quality improvement (QI) initiatives is often poor. Better guidance on "what good looks like" might help to mitigate some of the problems. We report a consensus-building process that sought to identify which features are important to include in QI measurement plans. METHODS: We conducted a three-stage consensus-building approach: (1) identifying the list of features of measurement plans that were potential candidates for inclusion based on literature review and the study team's experience; (2) a two-round modified Delphi exercise with a panel of experts to establish consensus on the importance of these features; and (3) a small in-person consensus group meeting to finalise the list of features. RESULTS: A list of 104 candidate questions was generated. A panel of 19 experts in the Delphi reviewed these questions and produced consensus on retaining 46 questions in the first round and on a further 22 in the second round. Thematic analysis of open text responses from the panellists suggested a number of areas of debate that were explicitly considered by the consensus group. The exercise yielded 74 questions (71% of 104) on which there was consensus in five categories of measurement relating to: design, data collection and management, analysis, action, and embedding. CONCLUSIONS: This study offers a consensus-based view on the features of a good measurement plan for a QI project in healthcare. The results may be of use to QI teams, funders and evaluators, but are likely to require further development and testing to ensure feasibility and usefulness.

Examining influences on speaking up among critical care healthcare providers in the United Arab Emirates.

OBJECTIVE: Assess perceived barriers to speaking up and to provide recommendations for reducing barriers to reporting adverse events and near misses. DESIGN, SETTING, PARTICIPANTS, INTERVENTION: A six-item survey was administered to critical care providers in 19 Intensive Care Units in Abu Dhabi as part of an organizational safety and quality improvement effort. MAIN OUTCOME MEASURES: Questions elicited perspectives about influences on reporting, perceived barriers and recommendations for conveying patient safety as an organizational priority. Qualitative thematic analyses were conducted for open-ended questions. RESULTS: A total of 1171 participants were invited to complete the survey and 639 responded (response rate = 54.6%). Compared to other stakeholders (e.g. the media, public), a larger proportion of respondents 'agreed/strongly agreed' that corporate health system leadership and the health regulatory authority encouraged and supported error reporting (83%; 75%), and had the most influence on their decisions to report (81%; 74%). 29.5% of respondents cited fear of repercussion as a barrier, and 21.3% of respondents indicated no barriers to reporting. Barriers included perceptions of a culture of blame and issues with reporting procedures. Recommendations to establish patient safety as an organizational priority included creating supportive environments to discuss errors, hiring staff to advocate for patient safety, and implementing policies to standardize clinical practices and streamline reporting procedures. CONCLUSIONS: Influences on reporting perceived by providers in the UAE were similar to those in the US and other countries. These findings highlight the roles of corporate leadership and regulators in developing non-punitive environments where reporting is a valuable and safe activity.

The PRIDx framework to engage payers in reducing diagnostic errors in healthcare.

OBJECTIVES: No framework currently exists to guide how payers and providers can collaboratively develop and implement incentives to improve diagnostic safety. We conducted a literature review and interviews with subject matter experts to develop a multi-component 'Payer Relationships for Improving Diagnoses (PRIDx)' framework, that could be used to engage payers in diagnostic safety efforts. CONTENT: The PRIDx framework, 1) conceptualizes diagnostic safety links to care provision, 2) illustrates ways to promote payer and provider engagement in the design and adoption of accountability mechanisms, and 3) explicates the use of data analytics. Certain approaches suggested by PRIDx were refined by subject matter expert interviewee perspectives. SUMMARY: The PRIDx framework can catalyze public and private payers to take specific actions to improve diagnostic safety. OUTLOOK: Implementation of the PRIDx framework requires new types of partnerships, including external support from public and private payer organizations, and requires creation of strong provider incentives without undermining providers' sense of professionalism and autonomy. PRIDx could help facilitate collaborative payer-provider approaches to improve diagnostic safety and generate research concepts, policy ideas, and potential innovations for engaging payers in diagnostic safety improvement activities.

The TeamSTEPPS for Improving Diagnosis Team Assessment Tool: Scale Development and Psychometric Evaluation.

INTRODUCTION: One in three patients is affected by diagnosis-related communication failures. Only a few valid and reliable instruments that measure teamwork and communication exist, and none of those focus on improving diagnosis. The authors developed, refined, and psychometrically evaluated the TeamSTEPPSⓇ for Improving Diagnosis Team Assessment Tool (TAT), which assesses diagnostic teamwork and communication in five critical teamwork domains and can be used to identify strengths and opportunities for improvement and monitor performance. METHODS: The TAT was administered as a cross-sectional survey to 360 health professionals across nine diverse US health systems. Content and construct validity were evaluated through pilot implementation and subject matter expert review. Reliability and internal consistency were assessed with Cronbach's alpha. To understand sources of variation in TAT scores and assess the tool's consistency across diverse health care organizations, generalizability theory (G-theory) was used. Best practices in screening for careless responding identified participants with random or nonvarying responses. RESULTS: Analyses indicated strong support for the tool. Content validity findings indicated that the TAT encompassed relevant diagnostic improvement teamwork and communication content. Construct validity, evaluated through pilot implementations, demonstrated the tool's effectiveness in assessing teamwork categories. Reliability analyses confirmed the TAT's internal consistency, with an overall Cronbach's alpha of 0.97. Each dimension of the TAT exhibited good reliability coefficients, ranging from 0.83 to 0.95. G-theory analysis showed that variations in TAT scores were primarily attributed to respondents (28.0%) and scale dimensions (59.6%); both are desirable facets of variation. Further, examination of careless respondents ensured the accuracy and quality of the results, enhancing the TAT's credibility as a valuable diagnostic improvement tool. CONCLUSION: Psychometric evaluation demonstrated that the TAT is a reliable and valid instrument for assessing teamwork and communication among and across diagnostic teams. The TAT adds a novel, evidence-based, psychometrically sound measurement tool to help advance diagnostic teamwork and communication to improve patient care and outcomes.

Impact of a national multimodal intervention to prevent catheter-related bloodstream infection in the ICU: the Spanish experience.

OBJECTIVE: Prevention of catheter-related bloodstream infection is a basic objective to optimize patient safety in the ICU. Building on the early success of a patient safety unit-based comprehensive intervention (the Keystone ICU project in Michigan), the Bacteremia Zero project aimed to assess its effectiveness after contextual adaptation at large-scale implementation in Spanish ICUs. DESIGN: Prospective time series. SETTING: A total of 192 ICUs throughout Spain. PATIENTS: All patients admitted to the participating ICUs during the study period (baseline April 1 to June 30, 2008; intervention period from January 1, 2009, to June 30, 2010). INTERVENTION: Engagement, education, execution, and evaluation were key program features. Main components of the intervention included a bundle of evidence-based clinical practices during insertion and maintenance of catheters and a unit-based safety program (including patient safety training and identification and analysis of errors through patient safety rounds) to improve the safety culture. MEASUREMENTS AND MAIN RESULTS: The number of catheter-related bloodstream infections was expressed as median and interquartile range. Poisson distribution was used to calculate incidence rates and risk estimates. The participating ICUs accounted for 68% of all ICUs in Spain. Catheter-related bloodstream infection was reduced after 16-18 months of participation (median 3.07 vs 1.12 episodes per 1,000 catheter-days, p<0.001). The adjusted incidence rate of bacteremia showed a 50% risk reduction (95% CI, 0.39-0.63) at the end of the follow-up period compared with baseline. The reduction was independent of hospital size and type. CONCLUSIONS: Results of the Bacteremia Zero project confirmed that the intervention significantly reduced catheter-related bloodstream infection after large-scale implementation in Spanish ICUs. This study suggests that the intervention can also be effective in different socioeconomic contexts even with decentralized health systems.

Using a logic model to design and evaluate quality and patient safety improvement programs.

Quality improvement programs often pose unique project management challenges, including multi-faceted interventions that evolve over time and teams with few resources for data collection. Thus, it is difficult to report methods and results. We developed a program to reduce central line-associated bloodstream infections (CLABSIs) and improve safety culture in intensive care units (ICUs). As previously reported, we worked with 103 Michigan ICUs to implement this program, and they achieved a 66% reduction in the median CLABSI rate and sustained the reduction. This success prompted the spread of this program to Spain, England, Peru and across the USA. We developed a logical framework approach (LFA) to guide project management; to incorporate the cultural, clinical and capacity variations among countries; and to ensure early alignment of the project's design and evaluation. In this paper, we describe the use of the LFA to systematically design, implement and evaluate large-scale, multi-faceted, quality improvement programs.

Surfacing safety hazards using standardized operating room briefings and debriefings at a large regional medical center.

BACKGROUND: Briefings and debriefings, previously shown to be a practical and feasible strategy to improve interdisciplinary communication and teamwork in the operating room (OR), was then assessed as a strategy to prospectively surface clinical and operational defects in surgical care--and thereby prevent patient harm. METHODS: A one-page, double-sided briefing and debriefing tool was used by surgical teams during cases at the William Beaumont Hospital Royal Oak (Royal Oak, Michigan) campus to surface clinical and operational defects during the study period (October 2006-May 2010). Defects were coded into six categories (with each category stratified by briefing or debriefing period) during the first six months, and refinement of coding resulted in expansion to 16 defect categories and no further stratification. A provider survey was used in January 2008 to interview a sample of 40 caregivers regarding the perceived effectiveness of the tool in surfacing defects. FINDINGS: The teams identified a total of 6,202 defects--an average of 141 defects per month--during the entire study period. Of 2,760 defects identified during the six-defect coding period, 1,265 (46%) surfaced during briefings, and the remaining 1,495 (54%) during debriefings. Equipment (48%) and communication (31%) issues were most prominent. Of 3,442 defects identified during the 16-defect coding period, the most common were Central Processing Department (CPD) instrumentation (22%) and Communication/Safety (15%). Overall, 70 (87%) of the 80 responses were in agreement that briefings were effective for surfacing defects, as were 59 (76%) of the 78 responses for debriefings. CONCLUSIONS: Briefings and debriefings were a practical and effective strategy to surface potential surgical defects in the operating rooms of a large medical center.

Using the Opportunity Estimator tool to improve engagement in a quality and safety intervention.

Teams throughout the United States participating in a program to reduce central line-associated bloodstream infections (CLABSIs) are using the Opportunity Estimator. This web-based tool translates CLABSI-related data into "opportunity estimates" of the patient lives and money that could be saved by reducing these infections.

A research framework for reducing preventable patient harm.

Programs to reduce central line-associated bloodstream infections (CLABSIs) have improved the safety of hospitalized patients. Efforts are underway to disseminate these successes broadly to reduce other types of hospital-acquired infectious and noninfectious preventable harms. Unfortunately, the ability to broadly measure and prevent other types of preventable harms, especially infectious harms, needs enhancement. Moreover, an overarching research framework for creating and integrating evidence will help expedite the development of national prevention programs. This article outlines a 5-phase translational (T) framework to develop robust research programs that reduce preventable harm, as follows: phase T0, discover opportunities and approaches to prevent adverse health care events; phase T1, use T0 discoveries to develop and test interventions on a small scale; phase T2, broaden and strengthen the evidence base for promising interventions to develop evidence-based guidelines; phase T3, translate guidelines into clinical practice; and phase T4, implement and evaluate T3 work on a national and international scale. Policy makers should use this framework to fill in the knowledge gaps, coordinate efforts among federal agencies, and prioritize research funding.

Assessing and improving safety climate in a large cohort of intensive care units.

OBJECTIVES: To evaluate the impact of a comprehensive unit-based safety program on safety climate in a large cohort of intensive care units participating in the Keystone intensive care unit project. DESIGN/SETTING: A prospective cohort collaborative study to improve quality of care and safety culture by implementing and evaluating patient safety interventions in intensive care units predominantly in the state of Michigan. INTERVENTIONS: The comprehensive unit-based safety program was the first intervention implemented by every intensive care unit participating in the collaborative. It is specifically designed to improve the various elements of a unit's safety culture, such as teamwork and safety climate. We administered the validated Safety Attitudes Questionnaire at baseline (2004) and after 2 yrs of exposure to the safety program (2006) to assess improvement. The safety climate domain on the survey includes seven items. MEASUREMENTS AND MAIN RESULTS: Post-safety climate scores for intensive care units. To interpret results, a score of <60% was in the "needs improvement" zone and a ≥10-point discrepancy in pre-post scores was needed to describe a difference. Hospital bed size, teaching status, and faith-based status were included in our analyses. Seventy-one intensive care units returned surveys in 2004 and 2006 with 71% and 73% response rates, respectively. Overall mean safety climate scores significantly improved from 42.5% (2004) to 52.2% (2006), t = -6.21, p < .001, with scores higher in faith-based intensive care units and smaller-bed-size hospitals. In 2004, 87% of intensive care units were in the "needs improvement" range and in 2006, 47% were in this range or did not score ≥10 points or higher. Five of seven safety climate items significantly improved from 2004 to 2006. CONCLUSIONS: A patient safety program designed to improve teamwork and culture was associated with significant improvements in overall mean safety climate scores in a large cohort of 71 intensive care units. Research linking improved climate scores and clinical outcomes is a critical next step.

Explaining Michigan: developing an ex post theory of a quality improvement program.

CONTEXT: Understanding how and why programs work-not simply whether they work-is crucial. Good theory is indispensable to advancing the science of improvement. We argue for the usefulness of ex post theorization of programs. METHODS: We propose an approach, located within the broad family of theory-oriented methods, for developing ex post theories of interventional programs. We use this approach to develop an ex post theory of the Michigan Intensive Care Unit (ICU) project, which attracted international attention by successfully reducing rates of central venous catheter bloodstream infections (CVC-BSIs). The procedure used to develop the ex post theory was (1) identify program leaders' initial theory of change and learning from running the program; (2) enhance this with new information in the form of theoretical contributions from social scientists; (3) synthesize prior and new information to produce an updated theory. FINDINGS: The Michigan project achieved its effects by (1) generating isomorphic pressures for ICUs to join the program and conform to its requirements; (2) creating a densely networked community with strong horizontal links that exerted normative pressures on members; (3) reframing CVC-BSIs as a social problem and addressing it through a professional movement combining "grassroots" features with a vertically integrating program structure; (4) using several interventions that functioned in different ways to shape a culture of commitment to doing better in practice; (5) harnessing data on infection rates as a disciplinary force; and (6) using "hard edges." CONCLUSIONS: Updating program theory in the light of experience from program implementation is essential to improving programs' generalizability and transferability, although it is not a substitute for concurrent evaluative fieldwork. Future iterations of programs based on the Michigan project, and improvement science more generally, may benefit from the updated theory present here.

Review article: high stakes and high risk: a focused qualitative review of hazards during cardiac surgery.

Cardiac surgery is a high-risk procedure performed by a multidisciplinary team using complex tools and technologies. Efforts to improve cardiac surgery safety have been ongoing for more than a decade, yet the literature provides little guidance regarding best practices for identifying errors and improving patient safety. This focused review of the literature was undertaken as part of the FOCUS initiative (Flawless Operative Cardiovascular Unified Systems), a multifaceted effort supported by the Society of Cardiovascular Anesthesiologists Foundation to identify hazards and develop evidence-based protocols to improve cardiac surgery safety. Hazards were defined as anything that posed a potential or real risk to the patient, including errors, near misses, and adverse events. Of the 1438 articles identified for title review, 390 underwent full abstract screening, and 69 underwent full article review, which in turn yielded 55 meeting the inclusion criteria for this review. Two key themes emerged. First, studies were predominantly reactive (responding to an event or report) instead of proactive (using prospective designs such as self-assessments and external reviewers, etc.) and very few tested interventions. Second, minor events were predictive of major problems: multiple, often minor, deviations from normal procedures caused a cascade effect, resulting in major distractions that ultimately led to major events. This review fills an important gap in the literature on cardiac surgery safety, that of systematically identifying and categorizing known hazards according to their primary systemic contributor (or contributors). We conclude with recommendations for improving patient outcomes by building a culture of safety, promoting transparency, standardizing training, increasing teamwork, and monitoring performance. Finally, there is an urgent need for studies that evaluate interventions to mitigate the inherent risks of cardiac surgery.

Prevention of central line-associated bloodstream infections: a journey toward eliminating preventable harm.

Central line-associated blood stream infections (CLABSI) are among the most common, lethal, and costly health care-associated infections. Recent large collaborative quality improvement efforts have achieved unprecedented and sustained reductions in CLABSI rates and demonstrate that these infections are largely preventable, even for exceedingly ill patients. The broad acceptance that zero CLABSI rates are an achievable goal has motivated and stimulated diverse groups of stakeholders, including public and private groups to develop policy tools and to mobilize their local constituents toward achieving this goal. Nevertheless, attributing reductions in CLABSI rates achieved by multifaceted quality improvement efforts solely to the use of checklists to ensure adherence with appropriate infection control practices is an easily made but crucial mistake. National CLABSI prevention is a shared responsibility and creating novel partnerships between government agencies, health care industry, and consumers is critical to making and sustaining progress in achieving the goals toward eliminating CLABSI.

Cardiac surgery errors: results from the UK National Reporting and Learning System.

OBJECTIVE: To describe cardiac surgery-related incidents and compare the types and severity of incidents occurring in the operating room (OR) versus non-OR locations. We hypothesized that the type and severity of incidents in cardiac surgery would differ in the OR compared with non-OR locations. DESIGN: A retrospective cross-sectional study of all incidents categorized as cardiac surgery in the UK National Reporting and Learning System database between January 2003 and February 2007. Differences in proportions were evaluated by χ(2) or Fischer's exact test. The odds ratio of an event occurring in the OR compared with all non-OR settings was calculated using logistic regression. The harm susceptibility ratio ranked locations by the degree of harm. SETTING: All trusts performing cardiac surgery. PARTICIPANTS: None. INTERVENTION: None. MAIN OUTCOME MEASURES: Cardiac surgery incidents occurring in the OR versus non-OR. RESULTS: A total of 4828 (<1%) incidents from 55 trusts were designated as involving cardiac surgery patients during the study period; 21% occurred in the OR. Overall, 32% of incidents resulted in harm: 23% of OR and 34% of non-OR incidents. The distribution of incident type and harmful incidents differed in the OR compared with the non-OR setting (P < 0.05). CONCLUSIONS: Our findings offer unique insights into the types of incidents occurring during cardiac surgical care in the UK. In the OR, interventions should focus on reducing errors associated with medical devices/equipment, whereas outside the OR, they may focus on medication errors and patient accidents.

Nursing leadership at the crossroads: evidence-based practice 'Matching Michigan-minimizing catheter related blood stream infections'(*).

AIMS AND OBJECTIVES: a highly successful intervention to reduce infections in intensive care units (ICUs) is now being widely replicated and involved significant nursing leadership. The objective of this manuscript is to describe briefly the intervention, and more explicitly the implications for nursing leadership as quality improvement and patient safety become global healthcare priorities. DESIGN: collaborative cohort study in over 100 ICUs in the United States to implement and evaluate interventions to improve patients' safety. METHODS: conceptual model aimed at improving clinicians' use of five evidence-based recommendations to reduce rates of catheter-related bloodstream infections rates, with measurement and feedback of infection rates. RESULTS: one hundred and three ICUs contributed 1981 ICU-months of data representing 375,757 catheter-days. The median rate of catheter-related bloodstream infection per 1000 catheter-days decreased from 2.7 infections at baseline to 0 at 3 months after implementation of the study intervention (P ≤ 0·002), and the mean rate per 1000 catheter-days decreased from 7.7 at baseline to 1.4 at 16-18 months of follow-up (P < 0·002). During the sustainability period, the mean bloodstream infection rate did not significantly change from the initial 18 month postimplementation period (-1%, 95% confidence interval -9% to 7%). Eighty seven percent of the original study participants had data available for the sustainability study. CONCLUSIONS: broad use of this intervention with achievement of similar results could substantially reduce the morbidity and costs associated with catheter-related bloodstream infections. RELEVANCE TO CLINICAL PRACTICE: the initial Michigan study and the follow-up analysis, that demonstrated sustained improvements, are leading to similar projects in other countries, include the Matching Michigan project in England. Discussing not only the technical components of the program, but also the nursing leadership aspects may assist nurses just embarking on this work.

Operational measurement of diagnostic safety: state of the science.

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Using evidence, rigorous measurement, and collaboration to eliminate central catheter-associated bloodstream infections.

Healthcare-associated infections are common, costly, and often lethal. Although there is growing pressure to reduce these infections, one project thus far has unprecedented collaboration among many groups at every level of health care. After this project produced a 66% reduction in central catheter-associated bloodstream infections and a median central catheter-associated bloodstream infection rate of zero across >100 intensive care units in Michigan, the Agency for Healthcare Research and Quality awarded a grant to spread this project to ten additional states. A program, called On the CUSP: Stop BSI, was formulated from the Michigan project, and additional funding from the Agency for Healthcare Research and Quality and private philanthropy has positioned the program for implementation state by state across the United States. Furthermore, the program is being implemented throughout Spain and England and is undergoing pilot testing in several hospitals in Peru. The model in this program balances the tension between being scientifically rigorous and feasible. The three main components of the model include translating evidence into practice at the bedside to prevent central catheter-associated bloodstream infections, improving culture and teamwork, and having a data collection system to monitor central catheter-associated bloodstream infections and other variables. If successful, this program will be the first national quality improvement program in the United States with quantifiable and measurable goals.

Improving the quality of quality improvement projects.

As the stakes grow for evaluating the quality of health care delivery, so too should greater attention be paid to the integrity of the design, conduct, and inferences made from QI projects. QI projects that seek to make inferences, especially public inferences, about the impact of an intervention to improve quality of care should be rigorously designed and evaluated, and limitations and potential biases transparently reported to understand how they may affect the conclusions suggested by the project. Our patients deserve nothing less.

An intervention to decrease catheter-related bloodstream infections in the ICU.

BACKGROUND: Catheter-related bloodstream infections occurring in the intensive care unit (ICU) are common, costly, and potentially lethal. METHODS: We conducted a collaborative cohort study predominantly in ICUs in Michigan. An evidence-based intervention was used to reduce the incidence of catheter-related bloodstream infections. Multilevel Poisson regression modeling was used to compare infection rates before, during, and up to 18 months after implementation of the study intervention. Rates of infection per 1000 catheter-days were measured at 3-month intervals, according to the guidelines of the National Nosocomial Infections Surveillance System. RESULTS: A total of 108 ICUs agreed to participate in the study, and 103 reported data. The analysis included 1981 ICU-months of data and 375,757 catheter-days. The median rate of catheter-related bloodstream infection per 1000 catheter-days decreased from 2.7 infections at baseline to 0 at 3 months after implementation of the study intervention (P< or =0.002), and the mean rate per 1000 catheter-days decreased from 7.7 at baseline to 1.4 at 16 to 18 months of follow-up (P<0.002). The regression model showed a significant decrease in infection rates from baseline, with incidence-rate ratios continuously decreasing from 0.62 (95% confidence interval [CI], 0.47 to 0.81) at 0 to 3 months after implementation of the intervention to 0.34 (95% CI, 0.23 to 0.50) at 16 to 18 months. CONCLUSIONS: An evidence-based intervention resulted in a large and sustained reduction (up to 66%) in rates of catheter-related bloodstream infection that was maintained throughout the 18-month study period.