Six Sigma - is it time to re-evaluate its value in laboratory medicine?

The Sigma metric is widely used in laboratory medicine.

QC Constellation: a cutting-edge solution for risk and patient-based quality control in clinical laboratories.

OBJECTIVES: Clinical laboratories face limitations in implementing advanced quality control (QC) methods with existing systems. This study aimed to develop a web-based application to addresses this gap, and improve QC practices. METHODS: QC Constellation, a web application built using Python 3.11, integrates various statistical QC modules. These include Levey-Jennings charts with Westgard rules, sigma-metric calculations, exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) charts, and method decision charts. Additionally, it offers a risk-based QC section and a patient-based QC module aligning with modern QC practices. The codes and the web application links for QC Constellation were shared at https://github.com/hikmetc/QC_Constellation, and http://qcconstellation.com, respectively. RESULTS: Using synthetic data, QC Constellation demonstrated effective implementation of Levey-Jennings charts with user-friendly features like checkboxes for Westgard rules and customizable moving averages graphs. Sigma-metric calculations for hypothetical performance values of serum total cholesterol were successfully performed using allowable total error and maximum allowable measurement uncertainty goals, and displayed on method decision charts. The utility of the risk-based QC module was exemplified by assessing QC plans for serum total cholesterol, showcasing the application's capability in calculating risk-based QC parameters including maximum unreliable final patient results, risk management index, and maximum run size and offering risk-based QC recommendations. Similarly, the patient-based QC and optimization modules were demonstrated using simulated sodium results. CONCLUSIONS: In conclusion, QC Constellation emerges as a pivotal tool for laboratory professionals, streamlining the management of quality control and analytical performance monitoring, while enhancing patient safety through optimized QC processes.

Evaluation of Sigma-Metric and Application of Quality Tools in Clinical Laboratory of a Tertiary Care Hospital.

Variability in analytical performance of some analyte indicated the need of evaluation of quality plan of our laboratory. We tried to put the same degree of effort into our quality metrics as we put into the laboratory processes themselves. Application of six sigma methodologies improve the quality by focusing on the root causes of the problems in performance and analyzing by flowcharts, fishbone diagrams and other quality tools. Sigma metric was calculated for laboratory parameters for a period of 8 months during 2018-19. The analytes with poor sigma metric were free Thyroxine (FT3, FT4), Sodium, Calcium and Magnesium. Sigma metric of free Thyroxine (FT3, FT4), Sodium, Calcium and Magnesium were below 3. A road map for process improvement was designed with DMAIC (Define-Measure-Analyze-Improve-Control) model to solve the issue. Possible causes for low analytical performance of the particular analytes were depicted in Fishbone diagram. The Fishbone analysis identified the water quality issues with electrolyte analysis while high ambient temperature was culprit for poor assay performance of free Thyroxine. Sigma metric of the analytical performance was assessed once again after root cause analysis. Sigmametric showed marked improvement in control phase. Identification of problems led to reduction in non value added work leading to adequate resource utilization by addressing the priority issue. Therefore, DMAIC tool with Fish bone model analysis can be recommended as a well suited method for troubleshooting in poor performance of laboratory parameter.

Novel quality control strategies for the determination of 25-hydroxyvitamin D by LC-MS/MS.

Mass spectrometry analysis has been applied in many important diagnostic fields of laboratory medicine. However, there is little literature to guide quality management systems for LC-MS/MS methods. In this study, LC-MS/MS 25-hydroxyvitamin D (25(OH)D) was used as an example to establish internal quality control strategies to ensure the accuracy of clinical vitamin D results. A total of 141 batches of samples were analyzed. Sample internal standard peak area variability, ion pair ratio, and physical examination population data were monitored as quality control strategies for 25(OH)D results. The analytical performance was evaluated by calculated Sigma metrics. Applying our quality control strategies, several abnormal data were monitored in the routine analysis. The daily peak area CV of 25(OH)D fluctuated within a certain range. By selecting P99 CV as the control target, two abnormal batches were found. The ratio of 25(OH)VD3 ion pairs was relatively stable. Among them, batch20230120 had a high CV value, which may be due to the bias caused by the limited number. According to the physical examination data, batch20220913 and batch20220919 exceeded the alarm limit. Sigma level of 25(OH)VD3 in the laboratory was 6.52, which achieved "excellent" performance. In conclusion, we established comprehensive quality control strategies for the determination of 25(OH)D by LC-MS/MS, which has high analytical performance and can provide more accurate reports for the clinic.

Sigma metric used to evaluate the performance of haematology analysers: choosing an internal reference analyser for the laboratory.

INTRODUCTION: The sigma metric offers a quantitative framework for evaluating process performance in clinical laboratories. This study aimed to evaluate the analytical performance of automated analysers in haematology laboratories, using the sigma metric to choose the best analyser as an internal reference analyser. MATERIALS AND METHODS: internal quality control (IQC) data were collected for 6 months from SNCS, and the sigma value was calculated for 9 haematology analysers in the laboratory. RESULTS: For the normal control level, a satisfactory mean sigma value ≥3 was observed for all of the studied parameters of all automated analysers. For the low control level, platelet (PLT) count by Instrument (Inst.) G performed poorly, with a mean sigma value <3. Inst. H, with all parameters' sigma values >4, performed best and was chosen as the internal reference analyser. CONCLUSION: The sigma metric can be used as a guide to choose the QC strategy and plan QC frequency. It can facilitate the comparison of the same assay performed by multiple systems.

Analysis of biochemical analytes using six sigma metrics with two analyzers at an Indian lab setting.

A zero defects goal was implemented in the clinical laboratory settings using a six-sigma model. Daily Internal Quality Control (IQC) and external quality control data from April-September 2023 was extracted to calculate the sigma metrics of 21 biochemical analytes based on Total Error Allowable (TEa), % bias and co-efficient of variation percent (CV%). A retrospective comparative study was conducted in the department of Clinical Biochemistry at Kanva Diagnostic Services Pvt. Ltd, Bengaluru, India. The analytical performance of the 21 biochemical analytes was tested on Cobas 6000 and C311 analyzers. Quality Goal Index (QGI) and root cause analysis was calculated to infer the reason for the deviation of six sigma. Method decision charts were plotted to show the comparison of the problem analytes on both the analyzers. On Cobas 6000 at level 1 IQC, out of 21 analytes, 10 analytes showed σ>6 and 10 analytes showed σ 3-6 and on C311, 15 analytes which showed σ>6 and 6 analytes that showed σ 3-6. On Cobas 6000 at level 2 IQC, out of 21 analytes, 12 analytes showed σ>6 and 8 analytes showed σ 3-6 and on C311 17 analytes showed σ>6 and 4 analytes showed σ 3-6. Creatinine failed to meet minimal sigma performance at both levels of IQC on Cobas 6000.

Comparison of Sigma metrics computed by three bias estimation approaches for 33 chemistry and 26 immunoassay analytes.

Objectives: Sigma metric can be calculated using a simple equation. However, there are multiple sources for the elements in the equation that may produce different Sigma values. This study aimed to investigate the importance of different bias estimation approaches for Sigma metric calculation. Methods: Sigma metrics were computed for 33 chemistry and 26 immunoassay analytes on the Roche Cobas 6000 analyzer. Bias was estimated by three approaches: (1) averaging the monthly bias values obtained from the external quality assurance (EQA) studies; (2) calculating the bias values from the regression equation derived from the EQA data; and (3) averaging the monthly bias values from the internal quality control (IQC) events. Sigma metrics were separately calculated for the two levels of the IQC samples using three bias estimation approaches. The resulting Sigma values were classified into five categories considering Westgard Sigma Rules as ≥6, <6 and ≥5, <5 and ≥4, <4 and ≥3, and <3. Results: When classifying Sigma metrics estimated by three bias estimation approaches for each assay, 16 chemistry assays at the IQC level 1 and 2 were observed to fall into different Sigma categories under at least one bias estimation approach. Similarly, for 12 immunoassays at the IQC level 1 and 2, Sigma category was different depending on bias estimation approach. Conclusions: Sigma metrics may differ depending on bias estimation approaches. This should be considered when using Six Sigma for assessing analytical performance or scheduling the IQC events.

Developing an evidence-based approach to quality control.

Effective Quality Control remains one of the pillars of Clinical Biochemistry. An understanding of the possible analytical errors that may occur, how to detect them efficiently and how to prevent them from causing patient harm are critical components of a Quality System. For some time, there have been questions about the theoretical basis of the models used to describe and detect analytical error. The current theory recognises two types of error, systematic and random and a system based on sampling the analytical process using a synthetic material to detect these errors. However, there are at least two other errors that are present. One is related to the QC material and the other, irregular errors. In this Opinion Piece, some of the underlying assumptions of Quality Control systems are described and analysed.

Performance specifications for sodium should not be based on biological variation.

When increasing the quality in clinical laboratories by decreasing measurement uncertainty, reliable methods are needed not only to quantify the performance of measuring systems, but also to set goals for the performance. Sigma metrics used in medical laboratories for documenting and expressing levels of performance, are evidently totally dependent on the "total permissible error" used in the formulas. Although the conventional biological variation (BV) based model for calculation of the permissible (or allowable) total error is commonly used, it has been shown to be flawed. Alternative methods are proposed, mainly also based on the within-subject BV. Measurement uncertainty models might offer an alternative to total error models. Defining the limits for analytical quality still poses a challenge in both models. The aim of the present paper is to critically discuss current methods for establishing performance specifications by using the measurement of sodium concentrations in plasma or serum. Sodium can be measured with high accuracy but fails by far to meet conventional performance specifications based on BV. Since the use of sodium concentrations is well established for supporting clinical care, we question the concept that quality criteria for sodium and similar analytes that are under strict homeostatic control are best set by biology.

Bias, the unfinished symphony.

In laboratory medicine, mathematical equations are frequently used to calculate various parameters including bias, imprecision, measurement uncertainty, sigma metric (SM), creatinine clearance, LDL-cholesterol concentration, etc. Mathematical equations have strict limitations and cannot be used in all situations and are not open to manipulations. Recently, a paper "Bias estimation for Sigma metric calculation: Arithmetic mean versus quadratic mean" was published in Biochemia Medica. In the paper, the author criticized the approach of taking the arithmetic mean of the multiple biases to obtain a single bias and proposed a quadratic method to estimate the overall bias using external quality assurance services (EQAS) data for SM calculation. This approach does not fit the purpose and it should be noted that using the correct equation in calculations is as important as using the correct reagent in the measurement of the analytes, therefore before using an equation, its suitability should be checked and confirmed.

Assessment of Sigma Metrics for Routine Chemistry Testing in 4 Laboratories in Kwa-Zulu Natal, South Africa.

BACKGROUND: Sigma metrics is a quantitative management tool. This study assessed the Six Sigma score for 26 chemistry analytes, compared scores with different total allowable errors (TEa) and use of scores for internal quality control (IQC) rules in 4 Laboratories in Kwa-Zulu Natal, South Africa. METHODS: Utilizing 6 months of IQC SD, CV, and bias data on albumin, alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, bicarbonate, calcium, total cholesterol, creatine kinase, chloride, creatinine, gamma glutamyl transferase, glucose, HDL-cholesterol, potassium, lactate dehydrogenase, magnesium, sodium, inorganic phosphate, direct bilirubin, total bilirubin, triglycerides, total protein, urea nitrogen, uric acid, and C-reactive protein (CRP) Six Sigma scores were calculated using Microsoft Excel 2016 and ideal IQC rules were determined. Six Sigma scores using Ricos et al. 2014, Royal College of Pathologists Australasia, and Clinical Laboratory Improvement Amendments TEas were compared. RESULTS: For levels 1, 2, and 3 respectively, analytes scoring >3 sigma was 9 (35%), 12 (46%), and 14 (54%) in Laboratory A; Laboratory B had 15 (58%), 19 (73%), and 17 (65%); Laboratory C had 12 (46%), 13 (50%), and 15 (58%); and Laboratory D had 13 (50%), 18 (69%), and 18 (69%). Albumin, calcium, sodium, magnesium, bicarbonate, and chloride scored <3; CRP scored >6 for all. In Laboratories A, B, C, and D, 7 (27%), 7 (27%), 6 (23%), and 8 (31%) analytes, respectively, required only 1 IQC rule. One of 21 analytes for Laboratories C and D, 3 for Laboratory A, and 0 for Laboratory B had the same sigma score with all 3 databases. CONCLUSION: Despite South Africa being a developing nation, many analytes are able to achieve >3 sigma.

Evaluation of Sigma metric approach for monitoring the performance of automated analyzers in hematology unit of Alexandria Main University Hospital.

INTRODUCTION: Sigma metric offers a quantitative framework for evaluating process performance in clinical laboratories. This study aimed to evaluate the analytical performance of automated analyzers in hematology unit of Alexandria Main University Hospital using the sigma metric approach. MATERIALS AND METHODS: Quality control data were collected for 6 months, and sigma value was calculated from hematology analyzers SYSMEX (XN 1000, XT 1800i), ADVIA (2120i, 2120), and coagulation analyzers SYSMEX CA 1500 (3610, 6336). RESULTS: For the normal control level, satisfactory mean sigma value ≥3 was observed for all of the studied parameters by all analyzers. For the high control level, red blood cell count by ADVIA 2120, and hematocrit by ADVIA (2120i and 2120) performed poorly with a mean sigma value <3. For the low control level, red blood cell count by ADVIA (2120i and 2120), hemoglobin by ADVIA 2120, hematocrit by ADVIA (2120i and 2120) and SYSMEX XN 1000, platelet count by the SYSMEX XT 1800i also performed poorly with a mean sigma value <3. Satisfactory mean sigma value of ≥3 was observed for prothrombin time and activated partial thromboplastin time for both normal and pathological control levels and analyzers. CONCLUSION: Sigma metrics can be used as a guide to make QC strategy and plan QC frequency and can facilitate the comparison of the same assay performance across multiple systems. Harmonization for TEa source is recommended to standardize sigma value calculation.

Internal Quality Control Data of Urine Reagent Strip Tests and Derivation of Control Rules Based on Sigma Metrics.

BACKGROUND: Urine reagent strip test (URST) results are semi-quantitative; therefore, the precision of URSTs is evaluated as the proportion of categorical results from repeated measurements of a sample that are concordant with an expected result. However, URSTs have quantitative readout values before ordinal results challenging statistical monitoring for internal quality control (IQC) with control rules. This study aimed to determine the sigma metric of URSTs and derive appropriate control rules for IQC. METHODS: The URiSCAN Super Plus fully automated urine analyzer (YD Diagnostics, Yongin, Korea) was used for URSTs. Change in reflectance rate (change %R) data from IQC for URSTs performed between November 2018 and May 2020 were analyzed. Red blood cells, bilirubin, urobilinogen, ketones, protein, glucose, leukocytes, and pH were measured from 2-3 levels of control materials. The total allowable error (TEa) for a grade was the difference in midpoints of a predefined change %R range between two adjacent grades. The sigma metric was calculated as TEa/SD. Sigma metric-based control rules were determined with Westgard EZ Rules 3 software (Westgard QC, Madison, WI, USA). RESULTS: Seven out of the eight analytes had a sigma metric >4 in the control materials with a negative grade (-), which were closer to the cut-offs. Corresponding control rules ranged from 12.5s to 13.5s. CONCLUSIONS: Although the URST is a semi-quantitative test, statistical IQC can be performed using the readout values. According to the sigma metric, control rules recommended for URST IQC in routine clinical practice are 12.5s to 13.5s.

Planning SQC strategies and adapting QC frequency for patient risk.

BACKGROUND: Risk-based Statistical QC strategies are recommended by the CLSI guidance for Statistical Quality Control (C24-Ed4). Using Parvin's patient risk model, QC frequency can be determined in terms of run size, i.e., the number of patient samples between QC events. Run size provides a practical goal for planning SQC strategies to achieve desired test reporting intervals. METHODS: A QC Frequency calculator is utilized to evaluate critical factors (quality required for test, precision and bias observed for method, rejection characteristics of SQC procedure) and also to consider patient risk as a variable for adjusting run size. RESULTS: We illustrate the planning of SQC strategies for a HbA1c test where two levels of controls show different sigma performance, for three different HbA1c analyzers used to achieve a common quality goal in a network of laboratories, and for an 18 test chemistry analyzer where a common run size is achieved by changes in control rules and adjustments for the patient risk of different tests. CONCLUSIONS: Run size provides a practical characteristic for adapting QC frequency to systematize the SQC strategies for multiple levels of controls or multiple tests in a chemistry analyzer. Patient risk can be an important variable for adapting run size to fit the laboratory's desired reporting intervals for high volume continuous production analyzers.

Six Sigma revisited: We need evidence to include a 1.5 SD shift in the extraanalytical phase of the total testing process.

The Six Sigma methodology has been widely implemented in industry, healthcare, and laboratory medicine since the mid-1980s. The performance of a process is evaluated by the sigma metric (SM), and 6 sigma represents world class performance, which implies that only 3.4 or less defects (or errors) per million opportunities (DPMO) are expected to occur. However, statistically, 6 sigma corresponds to 0.002 DPMO rather than 3.4 DPMO. The reason for this difference is the introduction of a 1.5 standard deviation (SD) shift to account for the random variation of the process around its target. In contrast, a 1.5 SD shift should be taken into account for normally distributed data, such as the analytical phase of the total testing process; in practice, this shift has been included in all type of calculations related to SM including non-normally distributed data. This causes great deviation of the SM from the actual level. To ensure that the SM value accurately reflects process performance, we concluded that a 1.5 SD shift should be used where it is necessary and formally appropriate. Additionally, 1.5 SD shift should not be considered as a constant parameter automatically included in all calculations related to SM.

Analytical performance assessment and improvement by means of the Failure mode and effect analysis (FMEA).

INTRODUCTION: Laboratories minimize risks through quality control but analytical errors still occur. Risk management can improve the quality of processes and increase patient safety. This study aims to use the failure mode and effect analysis (FMEA) to assess the analytical performance and measure the effectiveness of the risk mitigation actions implemented. MATERIALS AND METHODS: The measurands to be included in the study were selected based on the measurement errors obtained by participating in an External Quality Assessment (EQA) Scheme. These EQA results were used to perform an FMEA of the year 2017, providing a risk priority number that was converted into a Sigma value (σFMEA). A root-cause analysis was done when σFMEA was lower than 3. Once the causes were determined, corrective measures were implemented. An FMEA of 2018 was carried out to verify the effectiveness of the actions taken. RESULTS: The FMEA of 2017 showed that alkaline phosphatase (ALP) and sodium (Na) presented a σFMEA of less than 3. The FMEA of 2018 revealed that none of the measurands presented a σFMEA below 3 and that σFMEA for ALP and Na had increased. CONCLUSIONS: Failure mode and effect analysis is a useful tool to assess the analytical performance, solve problems and evaluate the effectiveness of the actions taken. Moreover, the proposed methodology allows to standardize the scoring of the scales, as well as the evaluation and prioritization of risks.

A novel Sigma metric encompasses global multi-site performance of 18 assays on the Abbott Alinity system.

OBJECTIVES: The Abbott Alinity family of chemistry and immunoassay systems recently launched with early adopters contributing imprecision and bias data, which was consolidated to assess the performance of Alinity assays across multiple sites using the Sigma metric. Multi-site Sigma metrics were determined for 3 ion-selective electrodes, 12 photometric assays, and 3 immunoassays across 11 independent laboratory sites in 9 countries. METHODS: Total allowable error (TEa) goals followed a previously defined hierarchy that used CLIA as the primary goal. Bias was calculated against the Abbott ARCHITECT system using Passing-Bablok regression analysis using individual site data or pooled aggregate data. Sigma metrics were calculated as (%TEa - |% bias|)/%CV. For individual-site analysis, the Sigma metrics for each assay were compared using the individual-site and the pooled biases. For multi-site analysis, the average CV and the pooled bias were used to generate a Pooled Sigma metric encompassing the global performance for a given assay. RESULTS: A total of 97 individual-site and 18 Pooled Sigma metrics were calculated for available assays. Individual Sigma metrics varied across sites, with 90% of assays performing 4 Sigma or higher, and 17 of 18 Pooled Sigma metrics indicated performance greater than 4 Sigma. Sigma metrics were significantly improved in 16 assays when using pooled bias rather than individual-site bias. CONCLUSIONS: This multi-center study applies a novel application of Sigma metrics to the first Alinity users and reveals analytical performance of greater than 4 Sigma for vast majority of assays. Laboratories with limited resources can leverage larger data sets for Pooled Sigma metric analysis, providing a tool to assess the consistency of analytical performance from multiple sites.

Sigma metric revisited: True known mistakes.

Six Sigma methodology has been used successfully in industry since the mid-1980s. Unfortunately, the same success has not been achieved in laboratory medicine. In this case, although the multidisciplinary structure of laboratory medicine is an important factor, the concept and statistical principles of Six Sigma have not been transferred correctly from industry to laboratory medicine. Furthermore, the performance of instruments and methods used in laboratory medicine is calculated by a modified equation that produces a value lower than the actual level. This causes unnecessary, increasing pressure on manufacturers in the market. We concluded that accurate implementation of the sigma metric in laboratory medicine is essential to protect both manufacturers by calculating the actual performance level of instruments, and patients by calculating the actual error rates.