A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken. METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts. RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021. CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving. METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available. CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Understanding the Differences in COVID-19 Case Fatality Rates Observed Across Alabama Counties.

To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.

Using Infant Mortality Data to Improve Maternal and Child Health Programs: An Application of Statistical Process Control Techniques for Rare Events.

Introduction The infant mortality rate (IMR) in the United States remains higher than most developed countries. To understand this public health issue and support state public health departments in displaying and analyzing data in ways that support learning, states participating in the Collaborative Improvement and Innovation Network to Reduce Infant Mortality (IM CoIIN) created statistical process control (SPC) charts for rare events. Methods State vital records data on live births and infant deaths was used to create U, T and G charts for Kansas and Alaska, two states participating in the IM CoIIN who sought methods to more effectively analyze IMR for subsets of their populations with infrequent number of deaths. The IMR and the number of days and number of births between infant deaths was charted for Kansas Non-Hispanic black population and six Alaska regions for the time periods 2013-2016 and 2011-2016, respectively. Established empirical patterns indicated points of special cause variation. Results The T and G charts for Kansas and G charts for Alaska depict points outside the upper control limit. These points indicate special cause variation and an increased number of days and/or births between deaths at these time periods. Discussion T and G charts offer value in examining rare events, and indicate special causes not detectable by U charts or other more traditional analytic methods. When small numbers make traditional analysis challenging, SPC has potential in the MCH field to better understand potential drivers of improvements in rare outcomes, inform decision making and take interventions to scale.

Learning from the design and development of the NHS Safety Thermometer.

QUALITY ISSUE: Research indicates that 10% of patients are harmed by healthcare but data that can be used in real time to improve safety are not routinely available. INITIAL ASSESSMENT: We identified the need for a prospective safety measurement system that healthcare professionals can use to improve safety locally, regionally and nationally. CHOICE OF SOLUTION: We designed, developed and implemented a national tool, named the NHS Safety Thermometer (NHS ST) with the goal of measuring the prevalence of harm from pressure ulcers, falls, urinary tract infection in patients with catheters and venous thromboembolism on one day each month for all NHS patients. IMPLEMENTATION: The NHS ST survey instrument was developed in a learning collaborative involving 161 organizations (e.g. hospitals and other delivery organizations) using a Plan, Do, Study, Act method. EVALUATION: Testing of operational definitions, technical capability and use were conducted and feedback systems were established by site coordinators in each participating organization. During the 17-month pilot, site coordinators reported a total of 73,651 patient entries. LESSONS LEARNED: It is feasible to obtain national data through standardized reporting by site coordinators at the point of care. Some caution is required in interpreting data and work is required locally to ensure data collection systems are robust and data collectors were trained. Sampling is an important strategy to optimize efficiency and reduce the burden of measurement.

An exploratory analysis of the model for understanding success in quality.

BACKGROUND: Experience suggests that differences in context produce variability in the effectiveness of quality improvement (QI) interventions. However, little is known about which contextual factors affect success or how they exert influence. PURPOSE: Using the Model for Understanding Success in Quality (MUSIQ), we perform exploratory quantitative tests of the role of context in QI success. METHODOLOGY: We used a cross-sectional design to survey individuals participating in QI projects in three settings: a pediatric hospital, hospitals affiliated with a state QI collaborative, and organizations sponsoring participants in an improvement advisor training program. Individuals participating in QI projects completed a questionnaire assessing contextual factors included in MUSIQ and measures of perceived success. Path analysis was used to test the direct, indirect, and total effects of context variables on QI success as hypothesized in MUSIQ. FINDINGS: In the 74 projects studied, most contextual factors in MUSIQ were found to be significantly related to at least one QI project performance outcome. Contextual factors exhibiting significant effects on two measures of perceived QI success included resource availability, QI team leadership, team QI skills, microsystem motivation, microsystem QI culture, and microsystem QI capability. There was weaker evidence for effects of senior leader project sponsors, organizational QI culture, QI team decision-making, and microsystem QI leadership. These initial tests add to the validity of MUSIQ as a tool for identifying which contextual factors affect improvement success and understanding how they exert influence. PRACTICE IMPLICATIONS: Using MUSIQ, managers and QI practitioners can begin to identify aspects of context that must be addressed before or during the execution of QI projects and plan strategies to modify context for increased success. Additional work by QI researchers to improve the theory, refine measurement approaches, and validate MUSIQ as a predictive tool in a wider range of QI efforts is necessary.

Challenging Cases in Statistical Process Control for Quality Improvement in Neonatal Intensive Care.

Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.

Developing Evidence for New Patient Safety Bundles Through Multihospital Collaboration.

OBJECTIVE: Multihospital collaboration for safety improvements is increasingly common, but strategies for developing bundles when effective evidence-based practices are not well described are limited. The Children's Hospitals' Solutions for Patient Safety (SPS) Network sought to further reduce patient harm by developing improvement bundles when preliminary evidence was limited. METHODS: As part of the novel Pioneer process, cohorts of volunteer SPS hospitals collaborated to identify a harm reduction bundle for carefully selected hospital-acquired harm categories where evidence-based practices were limited. For each harm type, a leadership team selected interventions (factors) for testing and guided the work throughout the Pioneer process. Using fundamental quality improvement techniques and a planned experimentation design, each participating hospital submitted outcome and process compliance data for the factor implemented. Data from all hospitals implementing that factor were analyzed together using Shewhart charts, response plots, and analysis of covariance to identify whether reliable implementation of the factor influenced outcomes. Factors were categorized based on strength of evidence and other clinical or evidentiary support. Factors with strong support were included in a final bundle and disseminated to all SPS hospitals. RESULTS: The SPS began the bundle identification process for nine harm types and three have completed the process. The analytic approach resulted in four scenarios that along with clinical input guided the inclusion or rejection of the factor in the final bundle. CONCLUSIONS: In this multihospital collaborative, quality improvement methods and planned experimentation were effective at developing evidence-based harm reduction bundles in situations where limited data for interventions exist.

Factorial Analysis Quantifies the Effects of Pediatric Discharge Bundle on Hospital Readmission.

BACKGROUND AND OBJECTIVES: Factorial design of a natural experiment was used to quantify the benefit of individual and combined bundle elements from a 4-element discharge transition bundle (checklist, teach-back, handoff to outpatient providers, and postdischarge phone call) on 30-day readmission rates (RRs). METHODS: A 24 factorial design matrix of 4 bundle element combinations was developed by using patient data (N = 7725) collected from January 2014 to December 2017 from 4 hospitals. Patients were classified into 3 clinical risk groups (CRGs): no chronic disease (CRG1), single chronic condition (CRG2), and complex chronic condition (CRG3). Estimated main effects of each bundle element and their interactions were evaluated by using Study-It software. Because of variation in subgroup size, important effects from the factorial analysis were determined by using weighted effect estimates. RESULTS: RR in CRG1 was 3.5% (n = 4003), 4.1% in CRG2 (n = 1936), and 17.6% in CRG3 (n = 1786). Across the 3 CRGs, the number of subjects in the factorial groupings ranged from 16 to 674. The single most effective element in reducing RR was the checklist in CRG1 and CRG2 (reducing RR by 1.3% and 3.0%) and teach-back in CRG3 (by 4.7%) The combination of teach-back plus a checklist had the greatest effect on reducing RR in CRG3 by 5.3%. CONCLUSIONS: The effect of bundle elements varied across risk groups, indicating that transition needs may vary on the basis of population. The combined use of teach-back plus a checklist had the greatest impact on reducing RR for medically complex patients.

A maturity grid assessment tool for learning networks.

BACKGROUND: The vision of learning healthcare systems (LHSs) is attractive as a more effective model for health care services, but achieving the vision is complex. There is limited literature describing the processes needed to construct such multicomponent systems or to assess development. METHODS: We used the concept of a capability maturity matrix to describe the maturation of necessary infrastructure and processes to create learning networks (LNs), multisite collaborative LHSs that use an actor-oriented network organizational architecture. We developed a network maturity grid (NMG) assessment tool by incorporating information from literature review, content theory from existing networks, and expert opinion to establish domains and components. We refined the maturity grid in response to feedback from network leadership teams. We followed NMG scores over time for nine LNs and plotted scores for each domain component with respect to SD for one participating network. We sought subjective feedback on the experience of applying the NMG to individual networks. RESULTS: LN leaders evaluated the scope, depth, and applicability of the NMG to their networks. Qualitative feedback from network leaders indicated that changes in NMG scores over time aligned with leaders' reports about growth in specific domains; changes in scores were consistent with network efforts to improve in various areas. Scores over time showed differences in maturation in the individual domains of each network. Scoring patterns, and SD for domain component scores, indicated consistency among LN leaders in some but not all aspects of network maturity. A case example from a participating network highlighted the value of the NMG in prompting strategic discussions about network development and demonstrated that the process of using the tool was itself valuable. CONCLUSIONS: The capability maturity grid proposed here provides a framework to help those interested in creating Learning Health Networks plan and develop them over time.

Using control charts to understand community variation in COVID-19.

Decision-makers need signals for action as the coronavirus disease 2019 (COVID-19) pandemic progresses. Our aim was to demonstrate a novel use of statistical process control to provide timely and interpretable displays of COVID-19 data that inform local mitigation and containment strategies. Healthcare and other industries use statistical process control to study variation and disaggregate data for purposes of understanding behavior of processes and systems and intervening on them. We developed control charts at the county and city/neighborhood level within one state (California) to illustrate their potential value for decision-makers. We found that COVID-19 rates vary by region and subregion, with periods of exponential and non-exponential growth and decline. Such disaggregation provides granularity that decision-makers can use to respond to the pandemic. The annotated time series presentation connects events and policies with observed data that may help mobilize and direct the actions of residents and other stakeholders. Policy-makers and communities require access to relevant, accurate data to respond to the evolving COVID-19 pandemic. Control charts could prove valuable given their potential ease of use and interpretability in real-time decision-making and for communication about the pandemic at a meaningful level for communities.

Joy and challenges in improving chronic illness care: capturing daily experiences of academic primary care teams.

BACKGROUND: Two chronic care collaboratives (The National Collaborative and the California Collaborative) were convened to facilitate implementing the chronic care model (CCM) in academic medical centers and into post-graduate medical education. OBJECTIVE: We developed and implemented an electronic team survey (ETS) to elicit, in real-time, team member's experiences in caring for people with chronic illness and the effect of the Collaborative on teams and teamwork. DESIGN: The ETS is a qualitative survey based on Electronic Event Sampling Methodology. It is designed to collect meaningful information about daily experience and any event that might influence team members' daily work and subsequent outcomes. PARTICIPANTS: Forty-one residency programs from 37 teaching hospitals participated in the collaboratives and comprised faculty and resident physicians, nurses, and administrative staff. APPROACH: Each team member participating in the collaboratives received an e-mail with directions to complete the ETS for four weeks during 2006 (the National Collaborative) and 2007 (the California Collaborative). KEY RESULTS: At the team level, the response rate to the ETS was 87% with team members submitting 1,145 narrative entries. Six key themes emerged from the analysis, which were consistent across all sites. Among teams that achieved better clinical outcomes on Collaborative clinical indicators, an additional key theme emerged: professional work satisfaction, or "Joy in Work". In contrast, among teams that performed lower in collaborative measures, two key themes emerged that reflected the effect of providing care in difficult institutional environments-"lack of professional satisfaction" and awareness of "system failures". CONCLUSIONS: The ETS provided a unique perspective into team performance and the day-to-day challenges and opportunities in chronic illness care. Further research is needed to explore systematic approaches to integrating the results from this study into the design of improvement efforts for clinical teams.

Developing measures of educational change for academic health care teams implementing the chronic care model in teaching practices.

BACKGROUND: The Chronic Care Model (CCM) is a multidimensional framework designed to improve care for patients with chronic health conditions. The model strives for productive interactions between informed, activated patients and proactive practice teams, resulting in better clinical outcomes and greater satisfaction. While measures for improving care may be clear, measures of residents' competency to provide chronic care do not exist. This report describes the process used to develop educational measures and results from CCM settings that used them to monitor curricular innovations. SUBJECTS: Twenty-six academic health care teams participating in the national and California Academic Chronic Care Collaboratives. METHOD: Using successive discussion groups and surveys, participants engaged in an iterative process to identify desirable and feasible educational measures for curricula that addressed educational objectives linked to the CCM. The measures were designed to facilitate residency programs' abilities to address new accreditation requirements and tested with teams actively engaged in redesigning educational programs. ANALYSIS: Field notes from each discussion and lists from work groups were synthesized using the CCM framework. Descriptive statistics were used to report survey results and measurement performance. RESULTS: Work groups generated educational objectives and 17 associated measurements. Seventeen (65%) teams provided feasibility and desirability ratings for the 17 measures. Two process measures were selected for use by all teams. Teams reported variable success using the measures. Several teams reported use of additional measures, suggesting more extensive curricular change. CONCLUSION: Using an iterative process in collaboration with program participants, we successfully defined a set of feasible and desirable education measures for academic health care teams using the CCM. These were used variably to measure the results of curricular changes, while simultaneously addressing requirements for residency accreditation.

Assessing Chronic Illness Care Education (ACIC-E): a tool for tracking educational re-design for improving chronic care education.

BACKGROUND: Recent Breakthrough Series Collaboratives have focused on improving chronic illness care, but few have included academic practices, and none have specifically targeted residency education in parallel with improving clinical care. Tools are available for assessing progress with clinical improvements, but no similar instruments have been developed for monitoring educational improvements for chronic care education. AIM: To design a survey to assist teaching practices with identifying curricular gaps in chronic care education and monitor efforts to address those gaps. METHODS: During a national academic chronic care collaborative, we used an iterative method to develop and pilot test a survey instrument modeled after the Assessing Chronic Illness Care (ACIC). We implemented this instrument, the ACIC-Education, in a second collaborative and assessed the relationship of survey results with reported educational measures. PARTICIPANTS: A combined 57 self-selected teams from 37 teaching hospitals enrolled in one of two collaboratives. ANALYSIS: We used descriptive statistics to report mean ACIC-E scores and educational measurement results, and Pearson's test for correlation between the final ACIC-E score and reported educational measures. RESULTS: A total of 29 teams from the national collaborative and 15 teams from the second collaborative in California completed the final ACIC-E. The instrument measured progress on all sub-scales of the Chronic Care Model. Fourteen California teams (70%) reported using two to six education measures (mean 4.3). The relationship between the final survey results and the number of educational measures reported was weak (R(2) = 0.06, p = 0.376), but improved when a single outlier was removed (R(2) = 0.37, p = 0.022). CONCLUSIONS: The ACIC-E instrument proved feasible to complete. Participating teams, on average, recorded modest improvement in all areas measured by the instrument over the duration of the collaboratives. The relationship between the final ACIC-E score and the number of educational measures was weak. Further research on its utility and validity is required.

A multi-institutional quality improvement initiative to transform education for chronic illness care in resident continuity practices.

BACKGROUND: There is a gap between the need for patient-centered, evidence-based primary care for the large burden of chronic illness in the US, and the training of resident physicians to provide that care. OBJECTIVE: To improve training for residents who provide chronic illness care in teaching practice settings. DESIGN: US teaching hospitals were invited to participate in one of two 18-month Breakthrough Series Collaboratives-either a national Collaborative, or a subsequent California Collaborative-to implement the Chronic Care Model (CCM) and related curriculum changes in resident practices. Most practices focused on patients with diabetes mellitus. Educational redesign strategies with related performance measures were developed for curricular innovations anchored in the CCM. In addition, three clinical measures-HbA1c <7%, LDL <100 mg/dL, and blood pressure

It Is Time to Reconsider Factorial Designs: How Bradford Hill and R. A. Fisher Shaped the Standard of Clinical Evidence.

OBJECTIVES: Could medical research and quality improvement studies be more productive with greater use of multifactor study designs? METHODS: Drawing on new primary sources and the literature, we examine the roles of A. Bradford Hill and Ronald A. Fisher in introducing the design of experiments in medicine. RESULTS: Hill did not create the randomized controlled trial, but he popularized the idea. His choice to set aside Fisher's advanced study designs shaped the development of clinical research and helped the single-treatment trial to become a methodological standard. CONCLUSIONS: Multifactor designs are not widely used in medicine despite their potential to make improvement initiatives and health services research more efficient and effective. Quality managers, health system leaders, and directors of research institutes could increase productivity and gain important insights by promoting a broader use of factorial designs to study multiple interventions simultaneously and to learn from interactions.

Orchestrated Testing of Formula Type to Reduce Length of Stay in Neonatal Abstinence Syndrome.

BACKGROUND: Despite the standardization of care, formula feeding varied across sites of the Ohio Perinatal Quality Collaborative (OPQC). We used orchestrated testing (OT) to learn from this variation and improve nonpharmacologic care of infants with neonatal abstinence syndrome (NAS) requiring pharmacologic treatment in Ohio. METHODS: To test the impact of formula on length of stay (LOS), treatment failure, and weight loss among infants hospitalized with NAS, we compared caloric content (high versus standard) and lactose content (low versus standard) using a 22 factorial design. During October 2015 to June 2016, OPQC sites joined 1 of 4 OT groups. We used response plots to examine the effect of each factor and control charts to track formula use and LOS. We used the OT results to revise the nonpharmacologic bundle and implemented it during 2017. RESULTS: Forty-seven sites caring for 546 NAS infants self-selected into the 4 OT groups. Response plots revealed the benefit of high-calorie formula (HCF) on weight loss, treatment failure, and LOS. The nonpharmacologic treatment bundle was updated to recommend HCF when breastfeeding was not possible. During implementation, HCF use increased, and LOS decreased from 17.1 to 16.4 days across the OPQC. CONCLUSIONS: OT revealed that HCF was associated with shorter LOS in OPQC sites. Implementation of a revised nonpharmacologic care bundle was followed by additional LOS improvement in Ohio. Despite some challenges in the implementation of OT, our findings support its usefulness for learning in improvement networks.

Addressing Challenges of Baseline Variability in the Clinical Setting: Lessons from an Emergency Department.

To demonstrate methods of adjusting data in quality improvement projects for better learning about interventions over time. METHODS: A secondary analysis of data from a quality improvement project to improve patient wait times at an urban academic pediatric emergency department using electronic medical data from 2015 to 2018. The primary outcome was the wait times for low-acuity patients. Control charts were used to determine if the interventions were effective in reducing wait times. Two different data adjustment techniques were applied to account for changes in patient volume and seasonal effects on the outcome measure. RESULTS: We more effectively demonstrated improved patient wait times after adjusting for patient volume or seasonality. Patient wait times decreased from 75.2 to 72.9 minutes after the intervention; a 3% decrease sustained over 18 months. A strong correlation between patient volume and wait times was noted. Process stability was achieved on the control charts after data adjustment, with one centerline shift after data adjustment in contrast to 5 centerline shifts required before data adjustment. CONCLUSION: Adjusting for seasonality or patient volume created process stability and improved learning from control charts. After adjustment, we sustained decreased patient wait times more than a year out from the original intervention Adjusting by patient volume seems to be a preferred method of adjustment. Our findings support the importance of adjusting for baseline variability affected by seasonality or patient volumes, especially in flow projects, as a high yield method for process improvement.