The value of Sigma-metrics in laboratory medicine.

While Six Sigma is used in different disciplines to improve quality, Tony Badric and Elvar Theodorsson in a recent paper in CCLM have questioned Six Sigma application in medical laboratory concluding Six Sigma has provided no value to medical laboratory. In addition, the authors have expanded their criticism to Total Analytical Error (TAE) model and statistical quality control. To address their arguments, we have explained the basics of TAE model and Six Sigma and have shown the value of Six Sigma to medical laboratory.

Intelligent Quality Management 2 with IntraSpect™ technology for quality control of GEM® Premier™ 5000 blood gas analyzers- A novel application of the patient sample as its own control.

Background: Quality control (QC) in point-of-care (POC) testing has been greatly improved by automatic control processes, such as the Intelligent Quality Management (iQM®) technology found in GEM Premier blood gas analyzers (Werfen, Bedford, MA). The 2nd generation technology, iQM2, provides additional capabilities, notably the incorporation of IntraSpect software that monitors the response curves of individual tests to detect transient errors caused by micro-clots, micro-bubbles or any event that disturbs the sensor response during sample data acquisition. IntraSpect is a novel form of patient-based, real-time quality control (PBRTQC). Methods: IntraSpect pattern recognition software monitors the last 15 measurements of each patient-response curve. Control limits for slope coefficients have been established from theoretical models and empirical data. Abnormal measurement behavior is flagged to identify transient errors that invalidate test results. Results from 1,013,391 patient samples were collected on 4,985 GEM Premier 5000 cartridges and 2,765 instruments in clinical use worldwide. Results and conclusions: Total pre-analytic and transient errors detected by IntraSpect were 1.91% worldwide. iQM2 with IntraSpect technology provides a unique control function detecting transient errors that would otherwise go undetected with traditional QC. Together with the statistical QC technology in iQM2, pre-analytic, analytic, and transient analytic errors are detected much faster-seconds versus hours-than by traditional statistical QC.

Multirule procedures vs moving average algorithms for IQC: An appropriate comparison reveals how best to combine their strengths.

BACKGROUND: Moving Average Algorithms (MAA) have been widely recommended for use in Patient Based Real Time Quality Control applications (PBRTQC) to supplement or replace traditional Internal Quality Control (IQC) techniques. A recent "proof of concept" study recommends applying MAAs to IQC data to replace traditional IQC procedures because they "outperform Westgard Rules," which is a current standard of practice for IQC. METHODS: We generated power curves for multi-rule procedures with 2 and 4 control measurements per QC event, as well as a Simple Moving Average having block sizes of 5, 10, and 20 control measurements. We also assessed time to detection in terms of the Average Number of QC Events required to detect different sizes of systematic errors. RESULTS: As expected, the more control measurements included in the control technique, the better the error detection. However, when QC performance is considered on the Sigma Scale, high Sigma methods require only 1 or 2 control measurements to detect medically important systematic errors. MAAs have very low ability to detect error at the first few QC events following shift, so they suffer a lag phase in detecting medically important errors. MAAs are most useful for methods having 4.0 Sigma performance or less. Even then, large systematic shifts are more quickly detected by simple single and multirule procedures. CONCLUSIONS: Choice of control techniques (rules, means, ranges, etc.) should consider the Sigma-metric of the method. For methods having Sigmas of 4 or greater, traditional single rule and multirule procedures with Ns up to 4 are most effective; below 4 Sigma, a multirule coupled with a Simple Moving Average (SMA) rule with Ns of 4 to 8 can improve error detection.

Internal quality control - past, present and future trends.

Objectives: This paper offers an historical view, through a summary of the internal quality control (IQC) models used from second half of twentyth century to those performed today and wants to give a projection on how the future should be addressed. Methods: The material used in this work study are all papers collected referring IQC procedures. The method used is the critical analysis of the different IQC models with a discussion on the weak and the strong points of each model. Results: First models were based on testing control materials and using multiples of the analytical procedure standard deviation as control limits. Later, these limits were substituted by values related with the intended use of test, mainly derived from biological variation. For measurands with no available control material methods based on replicate analysis of patient' samples were developed and have been improved recently; also, the sigma metrics that relates the quality desired with the laboratory performance has resulted in a highly efficient quality control model. Present tendency is to modulate IQC considering the workload and the impact of analytical failure in the patent harm. Conclusions: This paper remarks the strong points of IQC models, indicates the weak points that should be eliminated from practice and gives a future projection on how to promote patient safety through laboratory examinations.

A multi-test planning model for risk based statistical quality control strategies.

BACKGROUND: Efforts to improve QC for multi-test analytic systems should focus on risk-based bracketed SQC strategies, as recommended in the CLSI C24-Ed4 guidance for QC practices. The objective is to limit patient risk by controlling the expected number of erroneous patient test results that would be reported over the period an error condition goes undetected. METHODS: A planning model is described to provide a structured process for considering critical variables for the development of SQC strategies for continuous production multi-test analytic systems. The model aligns with the principles of the CLSI C24-Ed4 "roadmap" and calculation of QC frequency, or run size, based on Parvin's patient risk model. Calculations are performed using an electronic spreadsheet to facilitate application of the planning model. RESULTS: Three examples of published validation data are examined to demonstrate the application of the planning model for multi-test chemistry and enzyme analyzers. The ability to assess "what if" conditions is key to identifying the changes and improvements that are necessary to simplify the overall system to a manageable number of SQC procedures. CONCLUSIONS: The planning of risk based SQC strategies should align operational requirements for workload and reporting intervals with QC frequency in terms of the run size or the number of patient samples between QC events. Computer tools that support the calculation of run sizes greatly facilitate the planning process and make it practical for medical laboratories to quickly assess the effects of critical variables.

Planning SQC strategies and adapting QC frequency for patient risk.

BACKGROUND: Risk-based Statistical QC strategies are recommended by the CLSI guidance for Statistical Quality Control (C24-Ed4). Using Parvin's patient risk model, QC frequency can be determined in terms of run size, i.e., the number of patient samples between QC events. Run size provides a practical goal for planning SQC strategies to achieve desired test reporting intervals. METHODS: A QC Frequency calculator is utilized to evaluate critical factors (quality required for test, precision and bias observed for method, rejection characteristics of SQC procedure) and also to consider patient risk as a variable for adjusting run size. RESULTS: We illustrate the planning of SQC strategies for a HbA1c test where two levels of controls show different sigma performance, for three different HbA1c analyzers used to achieve a common quality goal in a network of laboratories, and for an 18 test chemistry analyzer where a common run size is achieved by changes in control rules and adjustments for the patient risk of different tests. CONCLUSIONS: Run size provides a practical characteristic for adapting QC frequency to systematize the SQC strategies for multiple levels of controls or multiple tests in a chemistry analyzer. Patient risk can be an important variable for adapting run size to fit the laboratory's desired reporting intervals for high volume continuous production analyzers.

Mistaken assumptions drive new Six Sigma model off the road.

Oosterhuis and Coskun recently proposed a new model for applying the Six Sigma concept to laboratory measurement processes. In criticizing the conventional Six Sigma model, the authors misinterpret the industrial basis for Six Sigma and mixup the Six Sigma "counting methodology" with the "variation methodology", thus many later attributions, conclusions, and recommendations are also mistaken. Although the authors attempt to justify the new model based on industrial principles, they ignore the fundamental relationship between Six Sigma and the process capability indices. The proposed model, the Sigma Metric is calculated as the ratio CVI/CVA, where CVI is individual biological variation and CVA is the observed analytical imprecision. This new metric does not take bias into account, which is a major limitation for application to laboratory testing processes. Thus, the new model does not provide a valid assessment of method performance, nor a practical methodology for selecting or designing statistical quality control procedures.

Establishing Evidence-Based Statistical Quality Control Practices.

OBJECTIVES: To establish an objective, scientific, evidence-based process for planning statistical quality control (SQC) procedures based on quality required for a test, precision and bias observed for a measurement procedure, probabilities of error detection and false rejection for different control rules and numbers of control measurements, and frequency of QC events (or run size) to minimize patient risk. METHODS: A Sigma-Metric Run Size Nomogram and Power Function Graphs have been used to guide the selection of control rules, numbers of control measurements, and frequency of QC events (or patient run size). RESULTS: A tabular summary is provided by a Sigma-Metric Run Size Matrix, with a graphical summary of Westgard Sigma Rules with Run Sizes. CONCLUSION: Medical laboratories can plan evidence-based SQC practices using simple tools that relate the Sigma-Metric of a testing process to the control rules, number of control measurements, and run size (or frequency of QC events).

Analytical Sigma metrics: A review of Six Sigma implementation tools for medical laboratories.

Sigma metrics have become a useful tool for all parts of the quality control (QC) design process. Through the allowable total error model of laboratory testing, analytical assay performance can be judged on the Six Sigma scale. This not only allows benchmarking the performance of methods and instruments on a universal scale, it allows laboratories to easily visualize performance, optimize the QC rules and numbers of control measurements they implement, and now even schedule the frequency of running those controls.

Selecting a Risk-Based SQC Procedure for a HbA1c Total QC Plan.

BACKGROUND: Recent US practice guidelines and laboratory regulations for quality control (QC) emphasize the development of QC plans and the application of risk management principles. The US Clinical Laboratory Improvement Amendments (CLIA) now includes an option to comply with QC regulations by developing an individualized QC plan (IQCP) based on a risk assessment of the total testing process. The Clinical and Laboratory Standards Institute (CLSI) has provided new practice guidelines for application of risk management to QC plans and statistical QC (SQC). METHODS: We describe an alternative approach for developing a total QC plan (TQCP) that includes a risk-based SQC procedure. CLIA compliance is maintained by analyzing at least 2 levels of controls per day. A Sigma-Metric SQC Run Size nomogram provides a graphical tool to simplify the selection of risk-based SQC procedures. APPLICATIONS: Current HbA1c method performance, as demonstrated by published method validation studies, is estimated to be 4-Sigma quality at best. Optimal SQC strategies require more QC than the CLIA minimum requirement of 2 levels per day. More complex control algorithms, more control measurements, and a bracketed mode of operation are needed to assure the intended quality of results. CONCLUSIONS: A total QC plan with a risk-based SQC procedure provides a simpler alternative to an individualized QC plan. A Sigma-Metric SQC Run Size nomogram provides a practical tool for selecting appropriate control rules, numbers of control measurements, and run size (or frequency of SQC). Applications demonstrate the need for continued improvement of analytical performance of HbA1c laboratory methods.

Planning Risk-Based SQC Schedules for Bracketed Operation of Continuous Production Analyzers.

BACKGROUND: To minimize patient risk, "bracketed" statistical quality control (SQC) is recommended in the new CLSI guidelines for SQC (C24-Ed4). Bracketed SQC requires that a QC event both precedes and follows (brackets) a group of patient samples. In optimizing a QC schedule, the frequency of QC or run size becomes an important planning consideration to maintain quality and also facilitate responsive reporting of results from continuous operation of high production analytic systems. METHODS: Different plans for optimizing a bracketed SQC schedule were investigated on the basis of Parvin's model for patient risk and CLSI C24-Ed4's recommendations for establishing QC schedules. A Sigma-metric run size nomogram was used to evaluate different QC schedules for processes of different sigma performance. RESULTS: For high Sigma performance, an effective SQC approach is to employ a multistage QC procedure utilizing a "startup" design at the beginning of production and a "monitor" design periodically throughout production. Example QC schedules are illustrated for applications with measurement procedures having 6-σ, 5-σ, and 4-σ performance. CONCLUSIONS: Continuous production analyzers that demonstrate high σ performance can be effectively controlled with multistage SQC designs that employ a startup QC event followed by periodic monitoring or bracketing QC events. Such designs can be optimized to minimize the risk of harm to patients.

Measuring Analytical Quality: Total Analytical Error Versus Measurement Uncertainty.

To characterize analytical quality of a laboratory test, common practice is to estimate Total Analytical Error (TAE) which includes both imprecision and trueness (bias). The metrologic approach is to determine Measurement Uncertainty (MU), which assumes bias can be eliminated, corrected, or ignored. Resolving the differences in these concepts and approaches is currently a global issue.

A Total Quality-Control Plan with Right-Sized Statistical Quality-Control.

A new Clinical Laboratory Improvement Amendments option for risk-based quality-control (QC) plans became effective in January, 2016. Called an Individualized QC Plan, this option requires the laboratory to perform a risk assessment, develop a QC plan, and implement a QC program to monitor ongoing performance of the QC plan. Difficulties in performing a risk assessment may limit validity of an Individualized QC Plan. A better alternative is to develop a Total QC Plan including a right-sized statistical QC procedure to detect medically important errors. Westgard Sigma Rules provides a simple way to select the right control rules and the right number of control measurements.

Six Sigma Quality Management System and Design of Risk-based Statistical Quality Control.

Six sigma concepts provide a quality management system (QMS) with many useful tools for managing quality in medical laboratories. This Six Sigma QMS is driven by the quality required for the intended use of a test. The most useful form for this quality requirement is the allowable total error. Calculation of a sigma-metric provides the best predictor of risk for an analytical examination process, as well as a design parameter for selecting the statistical quality control (SQC) procedure necessary to detect medically important errors. Simple point estimates of sigma at medical decision concentrations are sufficient for laboratory applications.

Planning Risk-Based Statistical Quality Control Strategies: Graphical Tools to Support the New Clinical and Laboratory Standards Institute C24-Ed4 Guidance.

BACKGROUND: Clinical and Laboratory Standards Institute (CLSI)'s new guideline for statistical quality control (SQC; C24-Ed4) (CLSI C24-Ed4, 2016; Parvin CA, 2017) recommends the implementation of risk-based SQC strategies. Important changes from earlier editions include alignment of principles and concepts with the general patient risk model in CLSI EP23A (CLSI EP23A, 2011) and a recommendation for optimizing the frequency of SQC (number of patients included in a run, or run size) on the basis of the expected number of unreliable final patient results. The guideline outlines a planning process for risk-based SQC strategies and describes 2 applications for examination procedures that provide 9-σ and 4-σ quality. A serious limitation is that there are no practical tools to help laboratories verify the results of these examples or perform their own applications. METHODS: Power curves that characterize the rejection characteristics of SQC procedures were used to predict the risk of erroneous patient results based on Parvin's MaxE(Nuf) parameter (Clin Chem 2008;54:2049-54). Run size was calculated from MaxE(Nuf) and related to the probability of error detection for the critical systematic error (Pedc). RESULTS: A plot of run size vs Pedc was prepared to provide a simple nomogram for estimating run size for common single-rule and multirule SQC procedures with Ns of 2 and 4. CONCLUSIONS: The "traditional" SQC selection process that uses power function graphs to select control rules and the number of control measurements can be extended to determine SQC frequency by use of a run size nomogram. Such practical tools are needed for planning risk-based SQC strategies.

Useful measures and models for analytical quality management in medical laboratories.

The 2014 Milan Conference "Defining analytical performance goals 15 years after the Stockholm Conference" initiated a new discussion of issues concerning goals for precision, trueness or bias, total analytical error (TAE), and measurement uncertainty (MU). Goal-setting models are critical for analytical quality management, along with error models, quality-assessment models, quality-planning models, as well as comprehensive models for quality management systems. There are also critical underlying issues, such as an emphasis on MU to the possible exclusion of TAE and a corresponding preference for separate precision and bias goals instead of a combined total error goal. This opinion recommends careful consideration of the differences in the concepts of accuracy and traceability and the appropriateness of different measures, particularly TAE as a measure of accuracy and MU as a measure of traceability. TAE is essential to manage quality within a medical laboratory and MU and trueness are essential to achieve comparability of results across laboratories. With this perspective, laboratory scientists can better understand the many measures and models needed for analytical quality management and assess their usefulness for practical applications in medical laboratories.