A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken. METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts. RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021. CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving. METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available. CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Understanding the Differences in COVID-19 Case Fatality Rates Observed Across Alabama Counties.

To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.

An exploratory analysis of the model for understanding success in quality.

BACKGROUND: Experience suggests that differences in context produce variability in the effectiveness of quality improvement (QI) interventions. However, little is known about which contextual factors affect success or how they exert influence. PURPOSE: Using the Model for Understanding Success in Quality (MUSIQ), we perform exploratory quantitative tests of the role of context in QI success. METHODOLOGY: We used a cross-sectional design to survey individuals participating in QI projects in three settings: a pediatric hospital, hospitals affiliated with a state QI collaborative, and organizations sponsoring participants in an improvement advisor training program. Individuals participating in QI projects completed a questionnaire assessing contextual factors included in MUSIQ and measures of perceived success. Path analysis was used to test the direct, indirect, and total effects of context variables on QI success as hypothesized in MUSIQ. FINDINGS: In the 74 projects studied, most contextual factors in MUSIQ were found to be significantly related to at least one QI project performance outcome. Contextual factors exhibiting significant effects on two measures of perceived QI success included resource availability, QI team leadership, team QI skills, microsystem motivation, microsystem QI culture, and microsystem QI capability. There was weaker evidence for effects of senior leader project sponsors, organizational QI culture, QI team decision-making, and microsystem QI leadership. These initial tests add to the validity of MUSIQ as a tool for identifying which contextual factors affect improvement success and understanding how they exert influence. PRACTICE IMPLICATIONS: Using MUSIQ, managers and QI practitioners can begin to identify aspects of context that must be addressed before or during the execution of QI projects and plan strategies to modify context for increased success. Additional work by QI researchers to improve the theory, refine measurement approaches, and validate MUSIQ as a predictive tool in a wider range of QI efforts is necessary.

Challenging Cases in Statistical Process Control for Quality Improvement in Neonatal Intensive Care.

Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.

A maturity grid assessment tool for learning networks.

BACKGROUND: The vision of learning healthcare systems (LHSs) is attractive as a more effective model for health care services, but achieving the vision is complex. There is limited literature describing the processes needed to construct such multicomponent systems or to assess development. METHODS: We used the concept of a capability maturity matrix to describe the maturation of necessary infrastructure and processes to create learning networks (LNs), multisite collaborative LHSs that use an actor-oriented network organizational architecture. We developed a network maturity grid (NMG) assessment tool by incorporating information from literature review, content theory from existing networks, and expert opinion to establish domains and components. We refined the maturity grid in response to feedback from network leadership teams. We followed NMG scores over time for nine LNs and plotted scores for each domain component with respect to SD for one participating network. We sought subjective feedback on the experience of applying the NMG to individual networks. RESULTS: LN leaders evaluated the scope, depth, and applicability of the NMG to their networks. Qualitative feedback from network leaders indicated that changes in NMG scores over time aligned with leaders' reports about growth in specific domains; changes in scores were consistent with network efforts to improve in various areas. Scores over time showed differences in maturation in the individual domains of each network. Scoring patterns, and SD for domain component scores, indicated consistency among LN leaders in some but not all aspects of network maturity. A case example from a participating network highlighted the value of the NMG in prompting strategic discussions about network development and demonstrated that the process of using the tool was itself valuable. CONCLUSIONS: The capability maturity grid proposed here provides a framework to help those interested in creating Learning Health Networks plan and develop them over time.

Using control charts to understand community variation in COVID-19.

Decision-makers need signals for action as the coronavirus disease 2019 (COVID-19) pandemic progresses. Our aim was to demonstrate a novel use of statistical process control to provide timely and interpretable displays of COVID-19 data that inform local mitigation and containment strategies. Healthcare and other industries use statistical process control to study variation and disaggregate data for purposes of understanding behavior of processes and systems and intervening on them. We developed control charts at the county and city/neighborhood level within one state (California) to illustrate their potential value for decision-makers. We found that COVID-19 rates vary by region and subregion, with periods of exponential and non-exponential growth and decline. Such disaggregation provides granularity that decision-makers can use to respond to the pandemic. The annotated time series presentation connects events and policies with observed data that may help mobilize and direct the actions of residents and other stakeholders. Policy-makers and communities require access to relevant, accurate data to respond to the evolving COVID-19 pandemic. Control charts could prove valuable given their potential ease of use and interpretability in real-time decision-making and for communication about the pandemic at a meaningful level for communities.

Joy and challenges in improving chronic illness care: capturing daily experiences of academic primary care teams.

BACKGROUND: Two chronic care collaboratives (The National Collaborative and the California Collaborative) were convened to facilitate implementing the chronic care model (CCM) in academic medical centers and into post-graduate medical education. OBJECTIVE: We developed and implemented an electronic team survey (ETS) to elicit, in real-time, team member's experiences in caring for people with chronic illness and the effect of the Collaborative on teams and teamwork. DESIGN: The ETS is a qualitative survey based on Electronic Event Sampling Methodology. It is designed to collect meaningful information about daily experience and any event that might influence team members' daily work and subsequent outcomes. PARTICIPANTS: Forty-one residency programs from 37 teaching hospitals participated in the collaboratives and comprised faculty and resident physicians, nurses, and administrative staff. APPROACH: Each team member participating in the collaboratives received an e-mail with directions to complete the ETS for four weeks during 2006 (the National Collaborative) and 2007 (the California Collaborative). KEY RESULTS: At the team level, the response rate to the ETS was 87% with team members submitting 1,145 narrative entries. Six key themes emerged from the analysis, which were consistent across all sites. Among teams that achieved better clinical outcomes on Collaborative clinical indicators, an additional key theme emerged: professional work satisfaction, or "Joy in Work". In contrast, among teams that performed lower in collaborative measures, two key themes emerged that reflected the effect of providing care in difficult institutional environments-"lack of professional satisfaction" and awareness of "system failures". CONCLUSIONS: The ETS provided a unique perspective into team performance and the day-to-day challenges and opportunities in chronic illness care. Further research is needed to explore systematic approaches to integrating the results from this study into the design of improvement efforts for clinical teams.

A multi-institutional quality improvement initiative to transform education for chronic illness care in resident continuity practices.

BACKGROUND: There is a gap between the need for patient-centered, evidence-based primary care for the large burden of chronic illness in the US, and the training of resident physicians to provide that care. OBJECTIVE: To improve training for residents who provide chronic illness care in teaching practice settings. DESIGN: US teaching hospitals were invited to participate in one of two 18-month Breakthrough Series Collaboratives-either a national Collaborative, or a subsequent California Collaborative-to implement the Chronic Care Model (CCM) and related curriculum changes in resident practices. Most practices focused on patients with diabetes mellitus. Educational redesign strategies with related performance measures were developed for curricular innovations anchored in the CCM. In addition, three clinical measures-HbA1c <7%, LDL <100 mg/dL, and blood pressure

It Is Time to Reconsider Factorial Designs: How Bradford Hill and R. A. Fisher Shaped the Standard of Clinical Evidence.

OBJECTIVES: Could medical research and quality improvement studies be more productive with greater use of multifactor study designs? METHODS: Drawing on new primary sources and the literature, we examine the roles of A. Bradford Hill and Ronald A. Fisher in introducing the design of experiments in medicine. RESULTS: Hill did not create the randomized controlled trial, but he popularized the idea. His choice to set aside Fisher's advanced study designs shaped the development of clinical research and helped the single-treatment trial to become a methodological standard. CONCLUSIONS: Multifactor designs are not widely used in medicine despite their potential to make improvement initiatives and health services research more efficient and effective. Quality managers, health system leaders, and directors of research institutes could increase productivity and gain important insights by promoting a broader use of factorial designs to study multiple interventions simultaneously and to learn from interactions.

Orchestrated Testing of Formula Type to Reduce Length of Stay in Neonatal Abstinence Syndrome.

BACKGROUND: Despite the standardization of care, formula feeding varied across sites of the Ohio Perinatal Quality Collaborative (OPQC). We used orchestrated testing (OT) to learn from this variation and improve nonpharmacologic care of infants with neonatal abstinence syndrome (NAS) requiring pharmacologic treatment in Ohio. METHODS: To test the impact of formula on length of stay (LOS), treatment failure, and weight loss among infants hospitalized with NAS, we compared caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design. During October 2015 to June 2016, OPQC sites joined 1 of 4 OT groups. We used response plots to examine the effect of each factor and control charts to track formula use and LOS. We used the OT results to revise the nonpharmacologic bundle and implemented it during 2017. RESULTS: Forty-seven sites caring for 546 NAS infants self-selected into the 4 OT groups. Response plots revealed the benefit of high-calorie formula (HCF) on weight loss, treatment failure, and LOS. The nonpharmacologic treatment bundle was updated to recommend HCF when breastfeeding was not possible. During implementation, HCF use increased, and LOS decreased from 17.1 to 16.4 days across the OPQC. CONCLUSIONS: OT revealed that HCF was associated with shorter LOS in OPQC sites. Implementation of a revised nonpharmacologic care bundle was followed by additional LOS improvement in Ohio. Despite some challenges in the implementation of OT, our findings support its usefulness for learning in improvement networks.

Addressing Challenges of Baseline Variability in the Clinical Setting: Lessons from an Emergency Department.

To demonstrate methods of adjusting data in quality improvement projects for better learning about interventions over time. METHODS: A secondary analysis of data from a quality improvement project to improve patient wait times at an urban academic pediatric emergency department using electronic medical data from 2015 to 2018. The primary outcome was the wait times for low-acuity patients. Control charts were used to determine if the interventions were effective in reducing wait times. Two different data adjustment techniques were applied to account for changes in patient volume and seasonal effects on the outcome measure. RESULTS: We more effectively demonstrated improved patient wait times after adjusting for patient volume or seasonality. Patient wait times decreased from 75.2 to 72.9 minutes after the intervention; a 3% decrease sustained over 18 months. A strong correlation between patient volume and wait times was noted. Process stability was achieved on the control charts after data adjustment, with one centerline shift after data adjustment in contrast to 5 centerline shifts required before data adjustment. CONCLUSION: Adjusting for seasonality or patient volume created process stability and improved learning from control charts. After adjustment, we sustained decreased patient wait times more than a year out from the original intervention Adjusting by patient volume seems to be a preferred method of adjustment. Our findings support the importance of adjusting for baseline variability affected by seasonality or patient volumes, especially in flow projects, as a high yield method for process improvement.

Improving the Context Supporting Quality Improvement in a Neonatal Intensive Care Unit Quality Collaborative: An Exploratory Field Study.

Successful quality improvement (QI) requires a supportive context. The goal was to determine whether a structured curriculum could help QI teams improve the context supporting their QI work. An exploratory field study was conducted of 43 teams participating in a neonatal intensive care unit QI collaborative. Using a curriculum based on the Model for Understanding Success in Quality, teams identified gaps in their context and tested interventions to modify context. Surveys and self-reflective journals were analyzed to understand how teams developed changes to modify context. More than half (55%) targeted contextual improvements within the microsystem, focusing on motivation and culture. "Information sharing" interventions to communicate information about the project as a strategy to engage more staff were the most common interventions tested. Further study is needed to determine if efforts to modify context consistently lead to greater outcome improvements.

Applying depression-specific change concepts in a collaborative breakthrough series.

BACKGROUND: Twenty ethnically and geographically diverse health care organizations, including 15 Bureau of Primary Health Care centers, participated in an Institute for Healthcare Improvement (IHI) collaborative Breakthrough Series (BTS) project on depression. Teams attended three learning sessions that emphasized the chronic illness care model, key depression change concepts, and how to initiate plan-do-study-act cycles. RESULTS: Seventeen of the 20 organizations completing the BTS achieved a faculty assessment of at least a 4 (5 indicates significant improvement). More than 2000 patients initiated depression treatment and were registered in the plan's depression registries. Patients in the centers who used the recommended measures had the following outcomes: 56% had significant change in their depressive symptoms at 12 weeks, 87% completed follow-up assessments, 54% continued antidepressant medication for at least 10 weeks, and 90% completed a structured diagnostic assessment before treatment. DISCUSSION: On the basis of the feedback from ten successful teams, the essential change concepts for depression were establishing and maintaining a patient registry, care coordination, diagnostic assessment, and proactive follow-up. Many of the BTS centers have continued to expand their depression treatment programs. The IHI BTS appears to be a viable method of disseminating evidence-based depression care.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

Seven propositions of the science of improvement: exploring foundations.

CONTEXT: The phrase "Science of Improvement" or "Improvement Science" is commonly used today by a range of people and professions to mean different things, creating confusion to those trying to learn about improvement. In this article, we briefly define the concepts of improvement and science, and review the history of the consideration of "improvement" as a science. METHODS: We trace key concepts and ideas in improvement to their philosophical and theoretical foundation with a focus on Deming's System of Profound Knowledge. We suggest that Deming's system has a firm association with many contemporary and historic philosophic and scientific debates and concepts. With reference to these debates and concepts, we identify 7 propositions that provide the scientific and philosophical foundation for the science of improvement. FINDINGS: A standard view of the science of improvement does not presently exist that is grounded in the philosophical and theoretical basis of the field. The 7 propositions outlined here demonstrate the value of examining the underpinnings of improvement. This is needed to both advance the field and minimize confusion about what the phrase "science of improvement" represents. We argue that advanced scientists of improvement are those who like Deming and Shewhart can integrate ideas, concepts, and models between scientific disciplines for the purpose of developing more robust improvement models, tools, and techniques with a focus on application and problem solving in real world contexts. CONCLUSIONS: The epistemological foundations and theoretical basis of the science of improvement and its reasoning methods need to be critically examined to ensure its continued development and relevance. If improvement efforts and projects in health care are to be characterized under the canon of science, then health care professionals engaged in quality improvement work would benefit from a standard set of core principles, a standard lexicon, and an understanding of the evolution of the science of improvement.

Reducing central line-associated bloodstream infections in North Carolina NICUs.

OBJECTIVE: Central lines in NICUs have long dwell times. Success in reducing central line-associated bloodstream infections (CLABSIs) requires a multidisciplinary team approach to line maintenance and insertion. The Perinatal Quality Collaborative of North Carolina (PQCNC) CLABSI project supported the development of NICU teams including parents, the implementation of an action plan with unique bundle elements and a rigorous reporting schedule. The goal was to reduce CLABSI rates by 75%. METHODS: Thirteen NICUs participated in an initiative developed over 3 months and deployed over 9 months. Teams participated in monthly webinars and quarterly face-to-face learning sessions. NICUs reported on bundle compliance and National Health Surveillance Network infection rates at baseline, during the intervention, and 3 and 12 months after the intervention. Process and outcome indicators were analyzed using statistical process control methods (SPC). RESULTS: Near-daily maintenance observations were requested for all lines with a 68% response rate. SPC analysis revealed a trend to an increase in bundle compliance. We also report significant adoption of a new maintenance bundle element, central line removal when enteral feedings reached 120 ml/kg per day. The PQCNC CLABSI rate decreased 71%, from 3.94 infections per 1000 line days to 1.16 infections per 1000 line days with sustainment 1 year later (P = .01). CONCLUSIONS: A collaborative structure targeting team development, family partnership, unique bundle elements and strict reporting on line care produced the largest reduction in CLABSI rates for any multiinstitutional NICU collaborative.