Benefits, limitations, and controversies on patient-based real-time quality control (PBRTQC) and the evidence behind the practice.

BACKGROUND: In recent years, there has been renewed interest in the "old" average of normals concept, now generally referred to as moving average quality control (MA QC) or patient-based real-time quality control (PBRTQC). However, there are some controversies regarding PBRTQC which this review aims to address while also indicating the current status of PBRTQC. CONTENT: This review gives the background of certain newly described optimization and validation methods. It also indicates how QC plans incorporating PBRTQC can be designed for greater effectiveness and/or (cost) efficiency. Furthermore, it discusses controversies regarding the complexity of obtaining PBRTQC settings, the replacement of iQC, and software functionality requirements. Finally, it presents evidence of the added value and practicability of PBRTQC. OUTLOOK: Recent developments in, and availability of, simulation methods to optimize and validate laboratory-specific PBRTQC procedures have enabled medical laboratories to implement PBRTQC in their daily practice. Furthermore, these methods have made it possible to demonstrate the practicability and added value of PBRTQC by means of two prospective "clinical" studies and other investigations. Although internal QC will remain an essential part of any QC plan, applying PBRTQC can now significantly improve its performance and (cost) efficiency.

Implementation of patient-based real-time quality control.

The quest to use patient results as quality control for routine clinical chemistry testing has long been driven by issues of the unavailability and cost of suitable quality control material and the matrix effects of synthetic material. Hematology laboratories were early adopters of average of normals techniques, primarily because of the difficulty in acquiring appropriate, stable quality control material, while in the chemistry laboratories, the perceived advantages and availability of synthetic material outweighed the disadvantages. However, the increasing volume of testing in clinical chemistry plus the capability of computer systems to deal with large and complex calculations has now made the use of patient-based quality control algorithms feasible. The desire to use patient-based quality control is also driven by increasing awareness that common quality control rules and frequency of analysis may fail to detect clinically significant assay biases. The non-commutability of quality control material has also become a problem as laboratories seek to harmonize results across regions and indeed globally. This review describes the history of patient-based quality control in clinical chemistry, summarizes the various approaches that can be implemented by laboratory professionals, and discusses how patient-based quality control can be integrated with traditional quality control techniques.

Understanding Patient-Based Real-Time Quality Control Using Simulation Modeling.

BACKGROUND: Patient-based real-time quality control (PBRTQC) avoids limitations of traditional quality control methods based on the measurement of stabilized control samples. However, PBRTQC needs to be adapted to the individual laboratories with parameters such as algorithm, truncation, block size, and control limit. METHODS: In a computer simulation, biases were added to real patient results of 10 analytes with diverse properties. Different PBRTQC methods were assessed on their ability to detect these biases early. RESULTS: The simulation based on 460 000 historical patient measurements for each analyte revealed several recommendations for PBRTQC. Control limit calculation with "percentiles of daily extremes" led to effective limits and allowed specification of the percentage of days with false alarms. However, changes in measurement distribution easily increased false alarms. Box-Cox but not logarithmic transformation improved error detection. Winsorization of outlying values often led to a better performance than simple outlier removal. For medians and Harrell-Davis 50 percentile estimators (HD50s), no truncation was necessary. Block size influenced medians substantially and HD50s to a lesser extent. Conversely, a change of truncation limits affected means and exponentially moving averages more than a change of block sizes. A large spread of patient measurements impeded error detection. PBRTQC methods were not always able to detect an allowable bias within the simulated 1000 erroneous measurements. A web application was developed to estimate PBRTQC performance. CONCLUSIONS: Computer simulations can optimize PBRTQC but some parameters are generally superior and can be taken as default.

Routine HIV Test Results in 6 US Clinical Laboratories Using the Recommended Laboratory HIV Testing Algorithm With Geenius HIV 1/2 Supplemental Assay.

BACKGROUND: Geenius HIV 1/2 Supplemental Assay (Geenius; Bio-Rad Laboratories) is the only Food and Drug Administration-approved HIV-1/HIV-2 antibody differentiation test for the second step in the HIV laboratory testing algorithm. We characterized the occurrence of true HIV-1 and HIV-2 infections as well as false results in 6 US clinical laboratories using Geenius. METHODS: We examined routine HIV testing outcome data from the time the laboratories began using the algorithm with Geenius until September 30, 2017. We calculated the positive predictive value for Geenius HIV-1 and HIV-2 reactivity separately. RESULTS: Of 5,046,684 specimens tested, 41,791 had reactive antigen/antibody test results. Most specimens with reactive antigen/antibody results were HIV-1 antibody-positive established infections (n = 32,421), 1,865 of which also had indeterminate HIV-2 bands present. Ninety-three specimens were HIV-2 antibody positive or untypable for HIV-1/HIV-2 antibody. Acute HIV-1 infections were found in 528 specimens; 881 specimens lacked the nucleic acid test to determine the possibility of acute HIV-1 infection. False-positive antigen/antibody test results were present in 7505 specimens. Few specimens (n = 363) had false-positive antigen/antibody results with indeterminate Geenius and negative HIV-1 nucleic acid test results. The positive predictive values of Geenius reactivity were 99.4% for HIV-1 and 4.3% for HIV-2. CONCLUSIONS: Routine testing using the laboratory testing algorithm with Geenius resulted in most specimens resolving as HIV negative or HIV-1 positive. The occurrence of indeterminate HIV-2 bands with a Geenius final assay interpretation of HIV-1 positive was more common than true HIV-2 infections. Reporting indeterminate HIV-2 results in this situation may cause confusion with interpreting HIV infection status.

Recommendation for performance verification of patient-based real-time quality control.

Patient-based real-time quality control (PBRTQC) is a laboratory tool for monitoring the performance of the testing process. It includes well-established procedures like Bull's algorithm, average of nomals, moving median, moving average (MA) and exponentially (weighted) MAs. Following the setup and optimization processes, a key step prior to the routine implementation of PBRTQC is the verification and documentation of the performance of the PBRTQC as part of the laboratory quality system. This verification process should provide a realistic representation of the performance of the PBRTQC in the environment it is being implemented in, to allow proper risk assessment by laboratory practitioners. This document focuses on the recommendation on performance verification of PBRTQC prior to implementation.

Patient-Based Real-Time Quality Control: Review and Recommendations.

For many years the concept of patient-based quality control (QC) has been discussed and implemented in hematology laboratories; however, the techniques have not been widely implemented in clinical chemistry. This is mainly because of the complexity of this form of QC, as it needs to be optimized for each population and often for each analyte. However, the clear advantages of this form of QC, together with the ongoing realization of the shortcomings of "conventional" QC, have driven a need to provide guidance to laboratories to assist in deploying patient-based QC. This overview describes the components of a patient-based QC system (calculation algorithm, block size, truncation limits, control limits) and the relationship of these to the analyte being controlled. We also discuss the need for patient-based QC system optimization using patient data from the individual testing laboratory to reliably detect systematic errors while ensuring that there are few false alarms. The term patient-based real-time quality control covers many activities that use data from patient samples to detect analytical errors. These activities include the monitoring of patient population parameters such as the mean or median analyte value or using single within-patient changes such as the delta check. In this report, we will restrict the discussion to population-based parameters. This overview is intended to serve as a guide for the implementation of a patient-based QC system. The report does not cover the clinical evaluation of the population.

Indirect methods for reference interval determination - review and recommendations.

Reference intervals are a vital part of the information supplied by clinical laboratories to support interpretation of numerical pathology results such as are produced in clinical chemistry and hematology laboratories. The traditional method for establishing reference intervals, known as the direct approach, is based on collecting samples from members of a preselected reference population, making the measurements and then determining the intervals. An alternative approach is to perform analysis of results generated as part of routine pathology testing and using appropriate statistical techniques to determine reference intervals. This is known as the indirect approach. This paper from a working group of the International Federation of Clinical Chemistry (IFCC) Committee on Reference Intervals and Decision Limits (C-RIDL) aims to summarize current thinking on indirect approaches to reference intervals. The indirect approach has some major potential advantages compared with direct methods. The processes are faster, cheaper and do not involve patient inconvenience, discomfort or the risks associated with generating new patient health information. Indirect methods also use the same preanalytical and analytical techniques used for patient management and can provide very large numbers for assessment. Limitations to the indirect methods include possible effects of diseased subpopulations on the derived interval. The IFCC C-RIDL aims to encourage the use of indirect methods to establish and verify reference intervals, to promote publication of such intervals with clear explanation of the process used and also to support the development of improved statistical techniques for these studies.

A study examining the bias of albumin and albumin/creatinine ratio measurements in urine.

BACKGROUND: The objective of the study was to examine the bias of albumin and albumin/creatinine (ACR) measurements in urine. METHODS: Pools of normal human urine were augmented with purified human serum albumin to generate a series of 12 samples covering the clinical range of interest for the measurement of ACR. Albumin and creatinine concentrations in these samples were analyzed three times on each of 3 days by 24 accredited laboratories in Canada and the USA. Reference values (RV) for albumin measurements were assigned by a liquid chromatography-tandem mass spectrometry (LC-MS/MS) comparative method and gravimetrically. Ten random urine samples (check samples) were analyzed as singlets and albumin and ACR values reported according to the routine practices of each laboratory. RESULTS: Augmented urine pools were shown to be commutable. Gravimetrically assigned target values were corrected for the presence of endogenous albumin using the LC-MS/MS comparative method. There was excellent agreement between the RVs as assigned by these two methods. All laboratory medians demonstrated a negative bias for the measurement of albumin in urine over the concentration range examined. The magnitude of this bias tended to decrease with increasing albumin concentrations. At baseline, only 10% of the patient ACR values met a performance limit of RV ± 15%. This increased to 84% and 86% following post-analytical correction for albumin and creatinine calibration bias, respectively. CONCLUSIONS: International organizations should take a leading role in the standardization of albumin measurements in urine. In the interim, accuracy based urine quality control samples may be used by clinical laboratories for monitoring the accuracy of their urinary albumin measurements.

Feasibility of using real-world free thyroxine data from the US and Europe to enable fast and efficient transfer of reference intervals from one population to another.

Objectives: The direct approach for determining reference intervals (RIs) is not always practical. This study aimed to generate evidence that a real-world data (RWD) approach could be applied to transfer free thyroxine RIs determined in one population to a second population, presenting an alternative to performing multiple RI determinations. Design and methods: Two datasets (US, n = 10,000; Europe, n = 10,000) were created from existing RWD. Descriptive statistics, density plots and cumulative distributions were produced for each data set and comparisons made. Cumulative probabilities at the lower and upper limits of the RIs were identified using an empirical cumulative distribution function. According to these probabilities, estimated percentiles for each dataset and estimated differences between the two sets of percentiles were obtained by case resampling bootstrapping. The estimated differences were then evaluated against a pre-determined acceptance criterion of ≤7.8% (inter-individual biological variability). The direct approach was used to validate the RWD approach. Results: The RWD approach provided similar descriptive statistics for both populations (mean: US = 16.1 pmol/L, Europe = 16.4 pmol/L; median: US = 15.4 pmol/L, Europe = 15.8 pmol/L). Differences between the estimated percentiles at the upper and lower limits of the RIs fulfilled the pre-determined acceptance criterion and the density plots and cumulative distributions demonstrated population homogeneity. Similar RI distributions were observed using the direct approach. Conclusions: This study provides evidence that a RWD approach can be used to transfer RIs determined in one population to another.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC).

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Performance criteria for testosterone measurements based on biological variation in adult males: recommendations from the Partnership for the Accurate Testing of Hormones.

BACKGROUND: Testosterone measurements that are accurate, reliable, and comparable across methodologies are crucial to improving public health. Current US Food and Drug Administration-cleared testosterone assays have important limitations. We sought to develop assay performance requirements on the basis of biological variation that allow physiologic changes to be distinguished from assay analytical errors. METHODS: From literature review, the technical advisory subcommittee of the Partnership for the Accurate Testing of Hormones compiled a database of articles regarding analytical and biological variability of testosterone. These data, mostly from direct immunoassay-based methodologies, were used to specify analytical performance goals derived from within- and between-person variability of testosterone. RESULTS: The allowable limits of desirable imprecision and bias on the basis of currently available biological variation data were 5.3% and 6.4%, respectively. The total error goal was 16.7%. From recent College of American Pathologists proficiency survey data, most currently available testosterone assays missed these analytical performance goals by wide margins. Data from the recently established CDC Hormone Standardization program showed that although the overall mean bias of selected certified assays was within 6.4%, individual sample measurements could show large variability in terms of precision, bias, and total error. CONCLUSIONS: Because accurate measurement of testosterone across a wide range of concentrations [approximately 2-2000 ng/dL (0.069-69.4 nmol/L)] is important, we recommend using available data on biological variation to calculate performance criteria across the full range of expected values. Additional studies should be conducted to obtain biological variation data on testosterone from women and children, and revisions should be made to the analytical goals for these patient populations.

Differences in the Distribution of IGF-I Concentrations Between European and US Populations.

Context: Method-specific reference intervals (RIs) determine utility of IGF-I as a biomarker in GH-related diseases. Differences between populations might affect applicability of RIs. Objective: To compare population-specific RIs derived from IGF-I routine testing in laboratories in the United States and Europe using the same assay. Design and setting: Uncensored routine IGF-I testing results generated over 5 years in 4 accredited laboratories (US, n = 778 173 males/710 752 females; Europe, n = 23 220 males/40 183 females). Main outcome measures: Construction of RIs by indirect statistical methods designed to use routine testing data (modified Hoffmann approach). Comparison to published RIs, between the US and Europe, and between regions in the United States with lower and higher mean body mass indexes (BMIs). Results: Lower limits (LLs) of RIs calculated from all routine data sets do not differ from the published LLs. The same is true for upper limits (ULs) calculated from European routine data. ULs derived from US routine data are significantly higher (children, 10-18 years [mean, %]: boys + 149.3 ng/mL [+34.6%]; girls + 94.9 ng/mL [+19.8%]); adults (19-95 years: males + 45 ng/mL [+20.3%]; and females + 29.7 ng/mL [+13.8%]). Average IGF-I is higher in samples from Colorado (lower mean BMI) compared with Alabama (P < 0.0001), although the difference is smaller than between each of them and Europe. Conclusions: We provide evidence that in large datasets from the same population, direct sampling and the indirect Hoffmann approach provide comparable RIs. Although LLs are comparable between Europe and the United States, the UL is significantly higher in the United States. We suggest use of adapted RIs for the United States.

Development of nation-wide reference intervals using an indirect method and harmonized assays.

OBJECTIVES: For many years, clinical laboratories have either verified or estimated reference intervals (RI) for laboratory tests. Those calculations have largely been performed by direct sampling analysis of ostensibly healthy individuals or by post-analysis biochemical screening. Recently however, indirect calculations have come to the forefront as an IFCC endorsed method by using normal and abnormal patient data. DESIGN AND METHODS: Using a large database of patient test results from Laboratory Corporation of America, age and gender based RIs, inclusive of neonatal, pediatric, and geriatric populations, were determined using a modified indirect method of Hoffmann, and represent a diverse population distributed across the United States from a nation-wide system of laboratories and is unbiased with respect to age, gender, race or geography. RESULTS: The tabulation of RIs using big data by an indirect method represent 72 M patient test results. The table includes 266 individual analytes consisting of approximately 2,700 age categories, including tests across multiple medical disciplines. CONCLUSIONS: To our knowledge, this is the largest collection of RIs that were calculated by an indirect method representing clinical chemistry, endocrinology, coagulation, and hematology analytes that have been derived with very powerful "Ns" for each age bracket. This process provides more robust RIs and allows for the determination of pediatric and geriatric RIs that would otherwise be difficult to obtain using traditional direct RI determinations.

Recommendations for laboratory informatics specifications needed for the application of patient-based real time quality control.

Patient based real time Quality Control (PBRTQC) algorithms provide many advantages over conventional QC approaches including lower cost, absence of commutability problems, continuous real-time monitoring of performance, and sensitivity to pre-analytical error. However, PBRTQC is not as simple to implement as conventional QC because of the requirement to access patient data as well as setting up appropriate rules, action protocols, and choosing best statistical algorithms. These requirements need capable and flexible laboratory informatics (middleware). In this document, the necessary features of software packages needed to support PBRTQC are discussed as well as recommendations for optimal integration of this technique into laboratory practice.

Establishing reference intervals for clinical laboratory test results: is there a better way?

Reference intervals are essential for clinical laboratory test interpretation and patient care. Methods for estimating them are expensive, difficult to perform, often inaccurate, and nonreproducible. A computerized indirect Hoffmann method was studied for accuracy and reproducibility. The study used data collected retrospectively for 5 analytes without exclusions and filtering from a nationwide chain of clinical reference laboratories in the United States. The accuracy was assessed by the comparability of reference intervals as calculated by the new method with published peer-reviewed studies, and reproducibility was assessed by the comparability of 2 sets of reference intervals derived from 2 different data sets. There was no statistically significant difference between the calculated and published reference intervals or between the 2 sets of intervals that were derived from different data sets. A computerized Hoffmann method for indirect estimation of reference intervals using stored test results is proved to be accurate and reproducible.