Quality control for serological testing.

OBJECTIVES: The quality control of serological assays remains controversial. The aim of this project was to describe the problems associated with a working model for controlling these assays and solutions, including using a source of well-defined targets and acceptable limits, a process to identify lot-to-lot reagent variation and an interpretation of the result that accounted for the clinical situation. False-negative results are problematic but can be reduced by identifying and comparing reagent lot variation with previous results. METHODS: The components of the Quality Assurance strategy are the following: Lot-to-lot reagent and calibrator variation assessment; dynamic, big-data approach to determine accurate targets and acceptable limits for manufacturer-provided QC material; negative QC monitoring process; use of commutable EQA with a sufficient method subgroup size to assess bias; clinical assessment of any statistically flagged error; and provision of support to the clinician for the interpretation of results. RESULTS: The model described has been used for twelve months, and acceptable variation has been maintained. CONCLUSIONS: The paper presents a solution that emphasizes the early detection of reagent lot variation and patient risk rather than instrument control. Reducing the risk of a false result to patients requires optimal assay quality control and an effective mechanism to support the clinician's use of these results in diagnosis and monitoring. The problems of serological assays are well-known, but there remain few integrated solutions in the literature.

Next-Generation Patient-Based Real-Time Quality Control Models.

Patient-based real-time QC (PBRTQC) uses patient-derived data to assess assay performance. PBRTQC algorithms have advanced in parallel with developments in computer science and the increased availability of more powerful computers. The uptake of Artificial Intelligence in PBRTQC has been rapid, with many stated advantages over conventional approaches. However, until this review, there has been no critical comparison of these. The PBRTQC algorithms based on moving averages, regression-adjusted real-time QC, neural networks and anomaly detection are described and contrasted. As Artificial Intelligence tools become more available to laboratories, user-friendly and computationally efficient, the major disadvantages, such as complexity and the need for high computing resources, are reduced and become attractive to implement in PBRTQC applications.

Six Sigma - is it time to re-evaluate its value in laboratory medicine?

The Sigma metric is widely used in laboratory medicine.

Enhanced patient-based real-time quality control using the graph-based anomaly detection.

OBJECTIVES: Patient-based real-time quality control (PBRTQC) is an alternative tool for laboratories that has gained increasing attention. Despite the progress made by using various algorithms, the problems of data volume imbalance between in-control and out-of-control results, as well as the issue of variation remain challenges. We propose a novel integrated framework using anomaly detection and graph neural network, combining clinical variables and statistical algorithms, to improve the error detection performance of patient-based quality control. METHODS: The testing results of three representative analytes (sodium, potassium, and calcium) and eight independent variables of patients (test date, time, gender, age, department, patient type, and reference interval limits) were collected. Graph-based anomaly detection network was modeled and used to generate control limits. Proportional and random errors were simulated for performance evaluation. Five mainstream PBRTQC statistical algorithms were chosen for comparison. RESULTS: The framework of a patient-based graph anomaly detection network for real-time quality control (PGADQC) was established and proven feasible for error detection. Compared with classic PBRTQC, the PGADQC showed a more balanced performance for both positive and negative biases. For different analytes, the average number of patient samples until error detection (ANPed) of PGADQC decreased variably, and reductions could reach up to approximately 95 % at a small bias of 0.02 taking calcium as an example. CONCLUSIONS: The PGADQC is an effective framework for patient-based quality control, integrating statistical and artificial intelligence algorithms. It improves error detection in a data-driven fashion and provides a new approach for PBRTQC from the data science perspective.

The European Organisation of External Quality Assurance Providers in Laboratory Medicine (EQALM) Statement: guidelines for publishing about interlaboratory comparison studies (PubILC).

Data and results from interlaboratory comparison (ILC) studies, external quality assessment (EQA) and proficiency testing (PT) activities are important and valuable contributions both to the further development of all disciplines of medical laboratory diagnostics, and to the evaluation and comparison of in vitro diagnostic assays. So far, however, there are no recommendations as to which essential items should be addressed in publications on interlaboratory comparisons. The European Organization of External Quality Assurance Providers in Laboratory Medicine (EQALM) recognized the need for such recommendations, and these were developed by a group of experts. The result of this endeavor is the EQALM Statement on items recommended to be addressed in publications on interlaboratory comparison activities (PubILC), in conjunction with a user-friendly checklist. Once adopted by authors and journals, the EQALM Statement will ensure essential information and/or study-related facts are included within publications on EQA/PT activities.

Sharing reference intervals and monitoring patients across laboratories - findings from a likely commutable external quality assurance program.

OBJECTIVES: Laboratory results are increasingly interpreted against common reference intervals (CRIs), published clinical decision limits, or previous results for the same patient performed at different laboratories. However, there are no established systems to determine whether current analytical performance justifies these interpretations. We analysed data from a likely commutable external quality assurance program (EQA) to assess these interpretations. METHODS: The use of CRIs was assessed by evaluating instrument group medians against minimum specifications for bias. The use of clinical decision limits was assessed using specifications from professional bodies, and the monitoring of patients by testing at different laboratories was assessed by comparing all-laboratory imprecision to within-subject biological variation. RESULTS: Five of the 18 analytes with Australasian CRIs did not meet specification for all instrument groups. Among these, calcium and magnesium failed for one instrument group out of seven, while bicarbonate, chloride, and lipase failed for two instrument groups. Of the 18 analytes reviewed currently without CRIs in Australasia, 10 candidates were identified. Among analytes with clinical decision limits, i.e. lipids, glucose, and vitamin D, only triglycerides met both bias and imprecision specifications, while vitamin D met the imprecision specification. Monitoring patients by testing at different laboratories was supported for 15 of the 46 (33 %) analyte-method principles groups that met minimum imprecision specifications. CONCLUSIONS: Analysis of data from commutable EQA programs can provide a mechanism for monitoring whether analytical performance justifies the interpretations made in contemporary laboratory practice. EQA providers should establish systems for routinely providing this information to the laboratory community.

External quality assurance in the era of standardization.

Metrology in clinical chemistry aims to ensure the equivalence of measurement results from different in-vitro diagnostic measurement devices (IVD MD) for use in healthcare. The metrological traceability of measurement results to higher-order references is the cornerstone to achieving equivalent results. However, other fundamentals are also needed, including the commutability of reference materials and external quality assessment (EQA) materials for monitoring the equivalence of measurement results at the end-user level. This manuscript summarizes the findings and opinions expressed at the Joint Community for Traceability in Laboratory Medicine (JCTLM) workshop held on December 4-5, 2023. The workshop explored the relationship between EQA/proficiency testing and metrological traceability to higher-order references. EQA monitors the equivalence of measurement results from end-user IVD MDs. The workshop discussed the role and challenges of using EQA to improve and maintain the equivalence of measurement results. It also elucidated current developments in establishing the clinical suitability of laboratory results expressed as analytical performance specifications (APS).

Investigating the Current Harmonization Status of Tumor Markers Using Global External Quality Assessment Programs: A Feasibility Study.

BACKGROUND: The harmonization status of most tumor markers (TMs) is unknown. We report a feasibility study performed to determine whether external quality assessment (EQA) programs can be used to obtain insights into the current harmonization status of the tumor markers α-fetoprotein (AFP), prostate specific antigen (PSA), carcinoembryonic antigen (CEA), cancer antigen (CA)125, CA15-3 and CA19-9. METHODS: EQA sample results provided by 6 EQA providers (INSTAND [Germany], Korean Association of External Quality Assessment Service [KEQAS, South Korea], National Center for Clinical Laboratories [NCCL, China], United Kingdom National External Quality Assessment Service [UK NEQAS, United Kingdom], Stichting Kwaliteitsbewaking Medische Laboratoriumdiagnostiek [SKML, the Netherlands], and the Royal College of Pathologists of Australasia Quality Assurance Programs [RCPAQAP, Australia]) between 2020 and 2021 were used. The consensus means, calculated from the measurement procedures present in all EQA programs (Abbott Alinity, Beckman Coulter DxI, Roche Cobas, and Siemens Atellica), was used as reference values. Per measurement procedure, the relative difference between consensus mean for each EQA sample and the mean of all patient-pool-based EQA samples were calculated and compared to minimum, desirable, and optimal allowable bias criteria based on biological variation. RESULTS: Between 19040 (CA15-3) and 25398 (PSA) individual results and 56 (PSA) to 76 (AFP) unique EQA samples were included in the final analysis. The mean differences with the consensus mean of patient-pool-based EQA samples for all measurement procedures were within the optimum bias criterion for AFP, the desirable bias for PSA, and the minimum bias criterion for CEA. However, CEA results <8 µg/L exceeded the minimum bias criterion. For CA125, CA15-3, and CA19-9, the harmonization status was outside the minimum bias criterion, with systematic differences identified. CONCLUSIONS: This study provides relevant information about the current harmonization status of 6 tumor markers. A pilot harmonization investigation for CEA, CA125, CA15-3, and CA19-9 would be desirable.

A model for managing quality control for a network of clinical chemistry instruments measuring the same analyte.

OBJECTIVES: Monitoring quality control for a laboratory or network with multiple instruments measuring the same analyte is challenging. We present a retrospective assessment of a method to detect medically significant out-of-control error conditions across a group of instruments measuring the same analyte. The purpose of the model was to ensure that results from any of several instruments measuring the same analytes in a laboratory or a network of laboratories provide comparable results and reduce patient risk. Limited literature has described how to manage QC in these very common situations. METHODS: Single Levey-Jennings control charts were designed using peer group target mean and control limits for five common clinical chemistry analytes in a network of eight analyzers in two different geographical sites. The QC rules used were 13s/22s/R4s, with the mean being a peer group mean derived from a large population of the same instrument and the same QC batch mean and a group CV. The peer group data used to set the target means and limits were from a quality assurance program supplied by the instrument supplier. Both statistical and clinical assessments of significance were used to evaluate QC failure. Instrument bias was continually monitored. RESULTS: It was demonstrated that the biases of each instrument were not statistically or clinically different compared to the peer group's average over six months from February 2023 until July 2023. Over this period, the error rate determined by the QC model was consistent with statistical expectations for the 13s/22s/R4s rule. There were no external quality assurance failures, and no detected error exceeded the TEa (medical impact). Thus, the combined statistical/clinical assessment reduced unnecessary recalibrations and the need to amend results. CONCLUSIONS: This paper describes the successful implementation of a quality control model for monitoring a network of instruments, measuring the same analytes and using externally provided quality control targets. The model continually assesses individual instrument bias and imprecision while ensuring all instruments in the network meet clinical goals for quality. The focus of this approach is on detecting medically significant out-of-control error conditions.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC).

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Clinical utility - Information about the usefulness of tests.

The clinical utility of a diagnostic test refers to its usefulness in improving patient outcomes, informing clinical decision-making, and optimizing healthcare resources. A diagnostic test with high clinical utility provides accurate, reliable, and actionable information that can guide appropriate treatment decisions, monitor treatment response, and identify potential adverse events or complications. Ultimately, the clinical utility of a diagnostic test depends on how well it can improve patient outcomes by guiding appropriate treatment decisions, improving clinical outcomes, and optimizing healthcare resource utilization. Healthcare providers need to weigh the benefits and drawbacks of using a particular diagnostic test in their clinical practice to determine its clinical utility.

Methods for determining clinical utility.

Measuring the clinical utility of a diagnostic test involves evaluating its impact on patient outcomes, clinical decision-making, and healthcare resource utilization. Determining clinical utility requires accessing patient medical history and outcomes data. These studies involve enrolling patients undergoing diagnostic tests and tracking their clinical outcomes. Researchers can determine the test's clinical utility by comparing the outcomes of patients who receive the diagnostic test to those who do not. These outcomes include benefits and harm. The highest level of evidence to support clinical utility determinations may be obtained from clinical trials. However, clinical laboratories are often not involved in clinical trials, and laboratory specialists may not be experienced in conducting such trials. Many established laboratory tests have never had clinical utility determined. Prospective studies assessing a diagnostic test's impact on clinical outcomes may require long-term patient monitoring, which is problematic. This paper presents methods that may be used to assess clinical utility.

Classification of "Near-patient" and "Point-of-Care" SARS-CoV-2 Nucleic Acid Amplification Test Systems and a first approach to evaluate their analytical independence of operator activities.

BACKGROUND: European legislation defines as "near-patient testing" (NPT) what is popularly and in other legislations specified as "point-of-care testing" (POCT). Systems intended for NPT/POCT use must be characterized by independence from operator activities during the analytic procedure. However, tools for evaluating this are lacking. We hypothesized that the variability of measurement results obtained from identical samples with a larger number of identical devices by different operators, expressed as the method-specific reproducibility of measurement results reported in External Quality Assessment (EQA) schemes, is an indicator for this characteristic. MATERIALS AND METHODS: Legal frameworks in the EU, the USA and Australia were evaluated about their requirements for NPT/POCT. EQA reproducibility of seven SARS-CoV-2-NAAT systems, all but one designated as "POCT", was calculated from variabilities in Ct values obtained from the respective device types in three different EQA schemes for virus genome detection. RESULTS: A matrix for characterizing test systems based on their technical complexity and the required operator competence was derived from requirements of the European In Vitro Diagnostic Regulation (IVDR) 2017/746. Good EQA reproducibility of the measurement results of the test systems investigated implies that different users in different locations have no recognizable influence on their measurement results. CONCLUSION: The fundamental suitability of test systems for NPT/POCT use according to IVDR can be easily verified using the evaluation matrix presented. EQA reproducibility is a specific characteristic indicating independence from operator activities of NPT/POCT assays. EQA reproducibility of other systems than those investigated here remains to be determined.

Machine Learning in Laboratory Medicine: Recommendations of the IFCC Working Group.

BACKGROUND: Machine learning (ML) has been applied to an increasing number of predictive problems in laboratory medicine, and published work to date suggests that it has tremendous potential for clinical applications. However, a number of groups have noted the potential pitfalls associated with this work, particularly if certain details of the development and validation pipelines are not carefully controlled. METHODS: To address these pitfalls and other specific challenges when applying machine learning in a laboratory medicine setting, a working group of the International Federation for Clinical Chemistry and Laboratory Medicine was convened to provide a guidance document for this domain. RESULTS: This manuscript represents consensus recommendations for best practices from that committee, with the goal of improving the quality of developed and published ML models designed for use in clinical laboratories. CONCLUSIONS: The committee believes that implementation of these best practices will improve the quality and reproducibility of machine learning utilized in laboratory medicine. SUMMARY: We have provided our consensus assessment of a number of important practices that are required to ensure that valid, reproducible machine learning (ML) models can be applied to address operational and diagnostic questions in the clinical laboratory. These practices span all phases of model development, from problem formulation through predictive implementation. Although it is not possible to exhaustively discuss every potential pitfall in ML workflows, we believe that our current guidelines capture best practices for avoiding the most common and potentially dangerous errors in this important emerging field.

Implementation of circulating tumour DNA multi-target mutation testing in plasma: a perspective from an external quality assessment providers' survey.

Demand for large-scale tumour profiling across cancer types has increased in recent years, driven by the emergence of targeted drug therapies. Analysing alternations in plasma circulating tumour DNA (ctDNA) for cancer detection can improve survival; ctDNA testing is recommended when tumour tissue is unavailable. An online survey of molecular pathology testing was circulated by six external quality assessment members of IQN Path to registered laboratories and all IQN Path collaborative corporate members. Data from 275 laboratories across 45 countries were collected; 245 (89%) perform molecular pathology testing, including 177 (64%) which perform plasma ctDNA diagnostic service testing. The most common tests were next-generation sequencing-based (n = 113). Genes with known stratified treatment options, including KRAS (n = 97), NRAS (n = 84), and EGFR (n = 130), were common targets. The uptake of ctDNA plasma testing and plans to implement further testing demonstrates the importance of support from a well-designed EQA scheme.