Challenging Cases in Statistical Process Control for Quality Improvement in Neonatal Intensive Care.

Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.

A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken. METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts. RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021. CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

A maturity grid assessment tool for learning networks.

BACKGROUND: The vision of learning healthcare systems (LHSs) is attractive as a more effective model for health care services, but achieving the vision is complex. There is limited literature describing the processes needed to construct such multicomponent systems or to assess development. METHODS: We used the concept of a capability maturity matrix to describe the maturation of necessary infrastructure and processes to create learning networks (LNs), multisite collaborative LHSs that use an actor-oriented network organizational architecture. We developed a network maturity grid (NMG) assessment tool by incorporating information from literature review, content theory from existing networks, and expert opinion to establish domains and components. We refined the maturity grid in response to feedback from network leadership teams. We followed NMG scores over time for nine LNs and plotted scores for each domain component with respect to SD for one participating network. We sought subjective feedback on the experience of applying the NMG to individual networks. RESULTS: LN leaders evaluated the scope, depth, and applicability of the NMG to their networks. Qualitative feedback from network leaders indicated that changes in NMG scores over time aligned with leaders' reports about growth in specific domains; changes in scores were consistent with network efforts to improve in various areas. Scores over time showed differences in maturation in the individual domains of each network. Scoring patterns, and SD for domain component scores, indicated consistency among LN leaders in some but not all aspects of network maturity. A case example from a participating network highlighted the value of the NMG in prompting strategic discussions about network development and demonstrated that the process of using the tool was itself valuable. CONCLUSIONS: The capability maturity grid proposed here provides a framework to help those interested in creating Learning Health Networks plan and develop them over time.

Using control charts to understand community variation in COVID-19.

Decision-makers need signals for action as the coronavirus disease 2019 (COVID-19) pandemic progresses. Our aim was to demonstrate a novel use of statistical process control to provide timely and interpretable displays of COVID-19 data that inform local mitigation and containment strategies. Healthcare and other industries use statistical process control to study variation and disaggregate data for purposes of understanding behavior of processes and systems and intervening on them. We developed control charts at the county and city/neighborhood level within one state (California) to illustrate their potential value for decision-makers. We found that COVID-19 rates vary by region and subregion, with periods of exponential and non-exponential growth and decline. Such disaggregation provides granularity that decision-makers can use to respond to the pandemic. The annotated time series presentation connects events and policies with observed data that may help mobilize and direct the actions of residents and other stakeholders. Policy-makers and communities require access to relevant, accurate data to respond to the evolving COVID-19 pandemic. Control charts could prove valuable given their potential ease of use and interpretability in real-time decision-making and for communication about the pandemic at a meaningful level for communities.

Understanding the Differences in COVID-19 Case Fatality Rates Observed Across Alabama Counties.

To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.

Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving. METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available. CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Orchestrated Testing of Formula Type to Reduce Length of Stay in Neonatal Abstinence Syndrome.

BACKGROUND: Despite the standardization of care, formula feeding varied across sites of the Ohio Perinatal Quality Collaborative (OPQC). We used orchestrated testing (OT) to learn from this variation and improve nonpharmacologic care of infants with neonatal abstinence syndrome (NAS) requiring pharmacologic treatment in Ohio. METHODS: To test the impact of formula on length of stay (LOS), treatment failure, and weight loss among infants hospitalized with NAS, we compared caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design. During October 2015 to June 2016, OPQC sites joined 1 of 4 OT groups. We used response plots to examine the effect of each factor and control charts to track formula use and LOS. We used the OT results to revise the nonpharmacologic bundle and implemented it during 2017. RESULTS: Forty-seven sites caring for 546 NAS infants self-selected into the 4 OT groups. Response plots revealed the benefit of high-calorie formula (HCF) on weight loss, treatment failure, and LOS. The nonpharmacologic treatment bundle was updated to recommend HCF when breastfeeding was not possible. During implementation, HCF use increased, and LOS decreased from 17.1 to 16.4 days across the OPQC. CONCLUSIONS: OT revealed that HCF was associated with shorter LOS in OPQC sites. Implementation of a revised nonpharmacologic care bundle was followed by additional LOS improvement in Ohio. Despite some challenges in the implementation of OT, our findings support its usefulness for learning in improvement networks.

It Is Time to Reconsider Factorial Designs: How Bradford Hill and R. A. Fisher Shaped the Standard of Clinical Evidence.

OBJECTIVES: Could medical research and quality improvement studies be more productive with greater use of multifactor study designs? METHODS: Drawing on new primary sources and the literature, we examine the roles of A. Bradford Hill and Ronald A. Fisher in introducing the design of experiments in medicine. RESULTS: Hill did not create the randomized controlled trial, but he popularized the idea. His choice to set aside Fisher's advanced study designs shaped the development of clinical research and helped the single-treatment trial to become a methodological standard. CONCLUSIONS: Multifactor designs are not widely used in medicine despite their potential to make improvement initiatives and health services research more efficient and effective. Quality managers, health system leaders, and directors of research institutes could increase productivity and gain important insights by promoting a broader use of factorial designs to study multiple interventions simultaneously and to learn from interactions.

Addressing Challenges of Baseline Variability in the Clinical Setting: Lessons from an Emergency Department.

To demonstrate methods of adjusting data in quality improvement projects for better learning about interventions over time. METHODS: A secondary analysis of data from a quality improvement project to improve patient wait times at an urban academic pediatric emergency department using electronic medical data from 2015 to 2018. The primary outcome was the wait times for low-acuity patients. Control charts were used to determine if the interventions were effective in reducing wait times. Two different data adjustment techniques were applied to account for changes in patient volume and seasonal effects on the outcome measure. RESULTS: We more effectively demonstrated improved patient wait times after adjusting for patient volume or seasonality. Patient wait times decreased from 75.2 to 72.9 minutes after the intervention; a 3% decrease sustained over 18 months. A strong correlation between patient volume and wait times was noted. Process stability was achieved on the control charts after data adjustment, with one centerline shift after data adjustment in contrast to 5 centerline shifts required before data adjustment. CONCLUSION: Adjusting for seasonality or patient volume created process stability and improved learning from control charts. After adjustment, we sustained decreased patient wait times more than a year out from the original intervention Adjusting by patient volume seems to be a preferred method of adjustment. Our findings support the importance of adjusting for baseline variability affected by seasonality or patient volumes, especially in flow projects, as a high yield method for process improvement.

Improving the Context Supporting Quality Improvement in a Neonatal Intensive Care Unit Quality Collaborative: An Exploratory Field Study.

Successful quality improvement (QI) requires a supportive context. The goal was to determine whether a structured curriculum could help QI teams improve the context supporting their QI work. An exploratory field study was conducted of 43 teams participating in a neonatal intensive care unit QI collaborative. Using a curriculum based on the Model for Understanding Success in Quality, teams identified gaps in their context and tested interventions to modify context. Surveys and self-reflective journals were analyzed to understand how teams developed changes to modify context. More than half (55%) targeted contextual improvements within the microsystem, focusing on motivation and culture. "Information sharing" interventions to communicate information about the project as a strategy to engage more staff were the most common interventions tested. Further study is needed to determine if efforts to modify context consistently lead to greater outcome improvements.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

Reducing central line-associated bloodstream infections in North Carolina NICUs.

OBJECTIVE: Central lines in NICUs have long dwell times. Success in reducing central line-associated bloodstream infections (CLABSIs) requires a multidisciplinary team approach to line maintenance and insertion. The Perinatal Quality Collaborative of North Carolina (PQCNC) CLABSI project supported the development of NICU teams including parents, the implementation of an action plan with unique bundle elements and a rigorous reporting schedule. The goal was to reduce CLABSI rates by 75%. METHODS: Thirteen NICUs participated in an initiative developed over 3 months and deployed over 9 months. Teams participated in monthly webinars and quarterly face-to-face learning sessions. NICUs reported on bundle compliance and National Health Surveillance Network infection rates at baseline, during the intervention, and 3 and 12 months after the intervention. Process and outcome indicators were analyzed using statistical process control methods (SPC). RESULTS: Near-daily maintenance observations were requested for all lines with a 68% response rate. SPC analysis revealed a trend to an increase in bundle compliance. We also report significant adoption of a new maintenance bundle element, central line removal when enteral feedings reached 120 ml/kg per day. The PQCNC CLABSI rate decreased 71%, from 3.94 infections per 1000 line days to 1.16 infections per 1000 line days with sustainment 1 year later (P = .01). CONCLUSIONS: A collaborative structure targeting team development, family partnership, unique bundle elements and strict reporting on line care produced the largest reduction in CLABSI rates for any multiinstitutional NICU collaborative.

Seven propositions of the science of improvement: exploring foundations.

CONTEXT: The phrase "Science of Improvement" or "Improvement Science" is commonly used today by a range of people and professions to mean different things, creating confusion to those trying to learn about improvement. In this article, we briefly define the concepts of improvement and science, and review the history of the consideration of "improvement" as a science. METHODS: We trace key concepts and ideas in improvement to their philosophical and theoretical foundation with a focus on Deming's System of Profound Knowledge. We suggest that Deming's system has a firm association with many contemporary and historic philosophic and scientific debates and concepts. With reference to these debates and concepts, we identify 7 propositions that provide the scientific and philosophical foundation for the science of improvement. FINDINGS: A standard view of the science of improvement does not presently exist that is grounded in the philosophical and theoretical basis of the field. The 7 propositions outlined here demonstrate the value of examining the underpinnings of improvement. This is needed to both advance the field and minimize confusion about what the phrase "science of improvement" represents. We argue that advanced scientists of improvement are those who like Deming and Shewhart can integrate ideas, concepts, and models between scientific disciplines for the purpose of developing more robust improvement models, tools, and techniques with a focus on application and problem solving in real world contexts. CONCLUSIONS: The epistemological foundations and theoretical basis of the science of improvement and its reasoning methods need to be critically examined to ensure its continued development and relevance. If improvement efforts and projects in health care are to be characterized under the canon of science, then health care professionals engaged in quality improvement work would benefit from a standard set of core principles, a standard lexicon, and an understanding of the evolution of the science of improvement.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

An exploratory analysis of the model for understanding success in quality.

BACKGROUND: Experience suggests that differences in context produce variability in the effectiveness of quality improvement (QI) interventions. However, little is known about which contextual factors affect success or how they exert influence. PURPOSE: Using the Model for Understanding Success in Quality (MUSIQ), we perform exploratory quantitative tests of the role of context in QI success. METHODOLOGY: We used a cross-sectional design to survey individuals participating in QI projects in three settings: a pediatric hospital, hospitals affiliated with a state QI collaborative, and organizations sponsoring participants in an improvement advisor training program. Individuals participating in QI projects completed a questionnaire assessing contextual factors included in MUSIQ and measures of perceived success. Path analysis was used to test the direct, indirect, and total effects of context variables on QI success as hypothesized in MUSIQ. FINDINGS: In the 74 projects studied, most contextual factors in MUSIQ were found to be significantly related to at least one QI project performance outcome. Contextual factors exhibiting significant effects on two measures of perceived QI success included resource availability, QI team leadership, team QI skills, microsystem motivation, microsystem QI culture, and microsystem QI capability. There was weaker evidence for effects of senior leader project sponsors, organizational QI culture, QI team decision-making, and microsystem QI leadership. These initial tests add to the validity of MUSIQ as a tool for identifying which contextual factors affect improvement success and understanding how they exert influence. PRACTICE IMPLICATIONS: Using MUSIQ, managers and QI practitioners can begin to identify aspects of context that must be addressed before or during the execution of QI projects and plan strategies to modify context for increased success. Additional work by QI researchers to improve the theory, refine measurement approaches, and validate MUSIQ as a predictive tool in a wider range of QI efforts is necessary.

Judgment sampling: a health care improvement perspective.

Sampling plays a major role in quality improvement work. Random sampling (assumed by most traditional statistical methods) is the exception in improvement situations. In most cases, some type of "judgment sample" is used to collect data from a system. Unfortunately, judgment sampling is not well understood. Judgment sampling relies upon those with process and subject matter knowledge to select useful samples for learning about process performance and the impact of changes over time. It many cases, where the goal is to learn about or improve a specific process or system, judgment samples are not merely the most convenient and economical approach, they are technically and conceptually the most appropriate approach. This is because improvement work is done in the real world in complex situations involving specific areas of concern and focus; in these situations, the assumptions of classical measurement theory neither can be met nor should an attempt be made to meet them. The purpose of this article is to describe judgment sampling and its importance in quality improvement work and studies with a focus on health care settings.

An unexpected increase in catheter-associated bloodstream infections at a children's hospital following introduction of the Spiros closed male connector.

BACKGROUND: Catheter-associated bloodstream infections (CA-BSIs) are associated with increased morbidity and mortality. Previous investigations have reported outbreaks of CA-BSI temporally associated with the use of needleless connector valves or similar devices. METHODS: We observed an unexpected increase in the rate of CA-BSI at our institution during August 2009. We used statistical process control and quality improvement methodology to identify the factor(s) associated with this increased rate of CA-BSI. RESULTS: We reviewed the overall hospital Shewhart U chart for CA-BSI, which indicated special cause variation with an unexpected cluster (6/9; 67%) of CA-BSIs localized to the oncology ward and the bone marrow transplant unit. An event-cause analysis review showed that 5 of these 9 infections were caused by Staphylococcus aureus. We discovered that the Spiros Closed Male Connector (ICU Medical, San Clemente, CA) had been introduced in these 2 units around the same time as the cluster of infections occurred. Based on this information, we discontinued the use of this device, and the CA-BSI rate and distribution of causative microorganisms returned to previous baseline values. CONCLUSION: This case study highlights the utility of statistical process control in the surveillance and investigation of CA-BSI.

The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement.

UNLABELLED: BACKGROUND Quality improvement (QI) efforts have become widespread in healthcare, however there is significant variability in their success. Differences in context are thought to be responsible for some of the variability seen. OBJECTIVE: To develop a conceptual model that can be used by organisations and QI researchers to understand and optimise contextual factors affecting the success of a QI project. METHODS: 10 QI experts were provided with the results of a systematic literature review and then participated in two rounds of opinion gathering to identify and define important contextual factors. The experts subsequently met in person to identify relationships among factors and to begin to build the model. RESULTS: The Model for Understanding Success in Quality (MUSIQ) is organised based on the level of the healthcare system and identifies 25 contextual factors likely to influence QI success. Contextual factors within microsystems and those related to the QI team are hypothesised to directly shape QI success, whereas factors within the organisation and external environment are believed to influence success indirectly. CONCLUSIONS: The MUSIQ framework has the potential to guide the application of QI methods in healthcare and focus research. The specificity of MUSIQ and the explicit delineation of relationships among factors allows a deeper understanding of the mechanism of action by which context influences QI success. MUSIQ also provides a foundation to support further studies to test and refine the theory and advance the field of QI science.