A hybrid Shewhart chart for visualizing and learning from epidemic data.

OBJECTIVE: As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over time. CONTEXT: Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken. METHODS: We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts. RESULTS: The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021. CONCLUSIONS: The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.

Understanding variation in reported covid-19 deaths with a novel Shewhart chart application.

OBJECTIVE: Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic. CONTEXT: Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change-rather than variability we would expect-care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving. METHODS: We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available. CONCLUSIONS: The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.

Government As Innovation Catalyst: Lessons From The Early Center For Medicare And Medicaid Innovation Models.

Congress established the Center for Medicare and Medicaid Innovation (CMMI) to design, test, and spread innovative payment and service delivery models that either reduce spending without reducing the quality of care or improve the quality of care without increasing spending. CMMI sought to leverage these models to foster market innovation and accelerate the transformation of payment and care delivery to achieve the Triple Aim of better health, better care, and lower cost. This article provides a perspective on the design and execution of CMMI's five initial models, the resulting outcomes and lessons, and how their core concepts evolved within and spread beyond CMMI. This experience yields three key insights that could inform future efforts by CMMI and public and private payers, including model designs and policy decisions. These insights center on the need for iterative testing and learning guided by market feedback, more realistic time frames to demonstrate impact on cost and quality, and greater integration of models.

Integrating Social Needs Into Health Care: A Twenty-Year Case Study Of Adaptation And Diffusion.

The US health care system has recently begun to account for patients' unmet social needs in care delivery and payment reform. This article presents a twenty-year qualitative case study of five stages of diffusion-testing and learning, standardization, replication, shifting from doing to enabling, and catalyzing broad adoption-of a practical approach for integrating social needs into clinical care. This case study of Health Leads and its funders confirms the importance of focusing on a clear aim, investing in model testing and standardization to enable subsequent responsiveness to the market, and the willingness of innovators and their investors to cede control of a model to allow local adaption and accelerate broad adoption.

Measuring Hospital-Wide Mortality-Pitfalls and Potential.

Risk-adjusted hospital-wide mortality has been proposed as a key indicator of system-level quality. Several risk-adjusted measures are available, and one-the hospital standardized mortality ratio (HSMR) - is publicly reported in a number of countries, but not in the United States. This paper reviews potential uses of such measures. We conclude that available methods are not suitable for interhospital comparisons or rankings and should not be used for pay-for-performance or value-based purchasing/payment. Hospital-wide mortality is a relatively imprecise, crude measure of quality, but disaggregation into condition- and service-line-specific mortality can facilitate targeted improvement efforts. If tracked over time, both observed and expected mortality rates should be monitored to ensure that apparent improvement is not due to increasing expected mortality, which could reflect changes in case mix or coding. Risk-adjusted mortality can be used as an initial signal that a hospital's mortality rate is significantly higher than statistically expected, prompting further inquiry.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

The "physician-led chart audit: " engaging providers in fortifying a culture of safety.

OBJECTIVES: Engaging physicians in quality and patient safety initiatives is a well-described challenge. Barriers include time constraints, lack of defined common purpose and leadership support, poorly communicated goals, and scarcity of supporting data (references 1, 2, 3).With reference to strengthening a culture of safety while meeting regulatory and performance standards, health-care systems face a difficult twin objective: educate the medical staff and its trainees and maintain high levels of compliance across inpatient, ambulatory clinic, and procedural areas.In 2010, our institution identified opportunities for improvement in physician performance related to several important patient safety standards. These issues had not been previously corrected by didactic sessions, written communication, and the chain of command. METHODS: To help address these general and site-specific problems, we developed and deployed a set of medical record audit tools entitled "physician-led audits" (PLAs). We trained providers in leadership positions to use the tools and to teach their use to others. We designed a system to capture data on frequency of use and results. The PLA process is distinctive in that it holds physicians accountable for the auditing and for follow-up afterward. RESULTS: With support from department chairs, division chiefs, residency program directors, and other leaders, close to 2000 PLAs were performed over a 10-month study period. CONCLUSIONS: The audits engaged physician leaders and the at-large medical staff, making them key participants in a system-wide improvement campaign. The tool was customized for use in a broad range of clinical settings and was widely and rapidly adopted, leading to important dialogue and a substantive contribution to our safety culture.

Health care quality improvement publication trends.

To analyze the extent of academic interest in quality improvement (QI) initiatives in medical practice, annual publication trends for the most well-known QI methodologies being used in health care settings were analyzed. A total of 10 key medical- and business-oriented library databases were examined: PubMed, Ovid MEDLINE, EMBASE, CINAHL, PsycINFO, ISI Web of Science, Scopus, the Cochrane Central Register of Controlled Trials, ABI/INFORM, and Business Source Complete. A total of 13 057 articles were identified that discuss at least 1 of 10 well-known QI concepts used in health care contexts, 8645 (66.2%) of which were classified as original research. "Total quality management" was the only methodology to demonstrate a significant decline in publication over time. "Continuous quality improvement" was the most common topic of study across all publication years, whereas articles discussing Lean methodology demonstrated the largest growth in publication volume over the past 2 decades. Health care QI publication volume increased substantially beginning in 1991.

Whole-patient measure of safety: using administrative data to assess the probability of highly undesirable events during hospitalization.

Hospitals often have limited ability to obtain primary clinical data from electronic health records to use in assessing quality and safety. We outline a new model that uses administrative data to gauge the safety of care at the hospital level. The model is based on a set of highly undesirable events (HUEs) defined using administrative data and can be customized to address the priorities and needs of different users. Patients with HUEs were identified using discharge abstracts from July 1, 2008 through June 30, 2010. Diagnoses were classified as HUEs based on the associated present-on-admission status. The 2-year study population comprised more than 6.5 million discharges from 161 hospitals. The proportion of hospitalizations including at least one HUE during the 24-month study period varied greatly among hospitals, with a mean of 7.74% (SD 2.3%) and a range of 13.32% (max, 15.31%; min, 1.99%). The whole-patient measure of safety provides a global measure to use in assessing hospitals with the patient's entire care experience in mind. As administrative and clinical datasets become more consistent, it becomes possible to use administrative data to compare the rates of HUEs across organizations and to identify opportunities for improvement.

Seven propositions of the science of improvement: exploring foundations.

CONTEXT: The phrase "Science of Improvement" or "Improvement Science" is commonly used today by a range of people and professions to mean different things, creating confusion to those trying to learn about improvement. In this article, we briefly define the concepts of improvement and science, and review the history of the consideration of "improvement" as a science. METHODS: We trace key concepts and ideas in improvement to their philosophical and theoretical foundation with a focus on Deming's System of Profound Knowledge. We suggest that Deming's system has a firm association with many contemporary and historic philosophic and scientific debates and concepts. With reference to these debates and concepts, we identify 7 propositions that provide the scientific and philosophical foundation for the science of improvement. FINDINGS: A standard view of the science of improvement does not presently exist that is grounded in the philosophical and theoretical basis of the field. The 7 propositions outlined here demonstrate the value of examining the underpinnings of improvement. This is needed to both advance the field and minimize confusion about what the phrase "science of improvement" represents. We argue that advanced scientists of improvement are those who like Deming and Shewhart can integrate ideas, concepts, and models between scientific disciplines for the purpose of developing more robust improvement models, tools, and techniques with a focus on application and problem solving in real world contexts. CONCLUSIONS: The epistemological foundations and theoretical basis of the science of improvement and its reasoning methods need to be critically examined to ensure its continued development and relevance. If improvement efforts and projects in health care are to be characterized under the canon of science, then health care professionals engaged in quality improvement work would benefit from a standard set of core principles, a standard lexicon, and an understanding of the evolution of the science of improvement.

Sampling considerations for health care improvement.

Sampling in improvement work can pose challenges. How is it different from the sampling strategies many use with research, clinical trials, or regulatory programs? What should improvement teams consider when determining a useful approach to sampling and a useful sample size? The aim of this article is to introduce some of the concepts related to sampling for improvement. We give specific guidance related to determining a useful sample size to a wider health care audience so that it can be applied to improvement projects in hospitals and health systems.

Large-scale improvement initiatives in healthcare: a scan of the literature.

CONTEXT: The goal of this article is to provide a succinct scan of the literature as it relates to the current thinking and practice in large-scale improvement initiatives in healthcare. METHOD: We employed a scan of the literature using a modified Delphi technique. A standard review form was used. The scan was limited to large-scale spread efforts in hospitals and healthcare systems. Each of the main factors that emerged during the scan was linked to secondary factors and organized using a driver diagram. FINDINGS: Four primary drivers (factors) emerged during our scan that inform large-scale change initiatives in healthcare: Planning and Infrastructure; Individual, Group, Organizational, and System Factors; The Process of Change; and Performance Measures and Evaluation. CONCLUSION: Our scan identified a tremendous amount of work being done around the world to improve healthcare. In general, our findings suggest these initiatives tend to be fragmented from an implementation standpoint. We identified primary and secondary drivers (factors) that can be used by those responsible for implementing large-scale improvement initiatives both at a strategy level and in their daily work. These drivers could serve as a "checklist" of ideas to consider in different testing and implementation situations.

Judgment sampling: a health care improvement perspective.

Sampling plays a major role in quality improvement work. Random sampling (assumed by most traditional statistical methods) is the exception in improvement situations. In most cases, some type of "judgment sample" is used to collect data from a system. Unfortunately, judgment sampling is not well understood. Judgment sampling relies upon those with process and subject matter knowledge to select useful samples for learning about process performance and the impact of changes over time. It many cases, where the goal is to learn about or improve a specific process or system, judgment samples are not merely the most convenient and economical approach, they are technically and conceptually the most appropriate approach. This is because improvement work is done in the real world in complex situations involving specific areas of concern and focus; in these situations, the assumptions of classical measurement theory neither can be met nor should an attempt be made to meet them. The purpose of this article is to describe judgment sampling and its importance in quality improvement work and studies with a focus on health care settings.

The epistemology of quality improvement: it's all Greek.

In Plato's Theaetetus, knowledge is defined as the intersection of truth and belief, where knowledge cannot be claimed if something is true but not believed or believed but not true. Using an example from neonatal intensive care, this paper adapts Plato's definition of the concept 'knowledge' and applies it to the field of quality improvement in order to explore and understand where current tensions may lie for both practitioners and decision makers. To increase the uptake of effective interventions, not only does there need to be scientific evidence, there also needs to be an understanding of how people's beliefs are changed in order to increase adoption more rapidly. Understanding how best to maximise the overlap between actual and best practice is where quality improvement needs to employ educational and social sciences' methodologies and techniques.

The run chart: a simple analytical tool for learning from variation in healthcare processes.

BACKGROUND: Those working in healthcare today are challenged more than ever before to quickly and efficiently learn from data to improve their services and delivery of care. There is broad agreement that healthcare professionals working on the front lines benefit greatly from the visual display of data presented in time order. AIM: To describe the run chart-an analytical tool commonly used by professionals in quality improvement but underutilised in healthcare. METHODS: A standard approach to the construction, use and interpretation of run charts for healthcare applications is developed based on the statistical process control literature. DISCUSSION: Run charts allow us to understand objectively if the changes we make to a process or system over time lead to improvements and do so with minimal mathematical complexity. This method of analyzing and reporting data is of greater value to improvement projects and teams than traditional aggregate summary statistics that ignore time order. Because of its utility and simplicity, the run chart has wide potential application in healthcare for practitioners and decision-makers. Run charts also provide the foundation for more sophisticated methods of analysis and learning such as Shewhart (control) charts and planned experimentation.

Organizational and leadership perceptions assessment: opportunities and challenges using the Baldrige companion surveys.

Combining the use of employee perception surveys with sound analytical techniques and models is critical to capturing high quality data from which effective decisions can be made in complex healthcare settings. This study used the Baldrige Award companion surveys with an analysis of variance (ANOVA) framework to identify discordant perceptions of hospital staff and leadership in the areas of customer focus, knowledge management, and results that were significant at the 0.05 and 0.01 levels. Senior leaders in the organization found the ANOVA framework helpful as they interpreted results from the Baldrige companion surveys and planned future improvement activities. During the execution of our study a number of difficulties and challenges arose that are not uncommon to survey administration in smaller settings, such as community hospitals, or in larger hospital settings with no research staff or research staff with relevant psychometric expertise. Our experience suggests that the Baldrige companion survey process would be enhanced by providing organizations with general guidance and protocols for optimal survey administration and data analysis. The purpose of this article is to outline the ANOVA model we used with the Baldrige companion surveys and to provide guidance related to the administration and analysis of these companion surveys for those that use them.

Balancing cost and precision in hospital accountability sampling.

The purpose of this article is to examine the relationship between sample size and cost using a benefit to cost ratio in the context of hospital accountability measures. We argue that the decision to use larger samples should include an assessment of the marginal benefit in terms of the cost to obtain the samples. Our main conclusion is that without recognizing and balancing the cost of different sampling schemes and developing reasonable cost limits we lose an opportunity at efficiency.