Assessment of accuracy of laboratory testing results, relative to peer group consensus values in external quality control, by bivariate z-score analysis: the example of D-Dimer.

OBJECTIVES: The aim of this study is to develop a practical method for bivariate z-score analysis which can be applied to the survey of an external quality assessment programme. METHODS: To develop the bivariate z-score analysis, the results of four surveys of the international D-Dimer external quality assessment programme of 2022 of the ECAT Foundation were used. The proposed methodology starts by identifying the bivariate outliers, using a Supervised Sequential Hotelling T2 control chart. The outlying data are removed, and all the remaining data are used to provide robust estimates of the parameters of the assumed underlying bivariate normal distribution. Based on these estimates two nested homocentric ellipses are drawn, corresponding to confidence levels of 95 and 99.7 %. The bivariate z-score plot described provides the laboratory with an indication of both systematic and random deviations from zero z-score values. The bivariate z-score analysis was examined within survey 2022-D4 across the three most frequently used methods. RESULTS: The number of z-score pairs included varied between 830 and 857 and the number of bivariate outliers varied between 20 and 28. The correlation between the z-score pairs varied between 0.431 and 0.647. The correlation between the z-score pairs for the three most frequently used varied between 0.208 and 0.636. CONCLUSIONS: The use of the bivariate z-score analysis is of major importance when multiple samples are distributed around in the same survey and dependency of the results is likely. Important lessons can be drawn from the shape of the ellipse with respect to random and systematic deviations, while individual laboratories have been informed about their position in the state-of-the-art distribution and whether they have to deal with systematic and/or random deviations.

Diagnosis of von Willebrand disease: An assessment of the quality of testing in North American laboratories.

BACKGROUND: Laboratory diagnosis of von Willebrand Disease (VWD) is complex. Reliance on laboratory testing can be problematic as different VWD screening panels, assays and methodologies can produce analytic variability in test results. OBJECTIVES: To compare the degree of imprecision among the VWD assays and within the platelet binding activity (PBA) assays, to determine the consensus among the VWD assays for correct classification of sample results, and to determine consensus among laboratories' von Willebrand factor (VWF) multimer interpretations and final interpretations of the VWD panels. PATIENTS/METHODS: Proficiency testing results from the North American Specialized Coagulation Laboratory Association (NASCOLA) submitted by laboratories from 2010 to 2019 for all normal, type (T) 1 VWD and T2 VWD samples were analysed. RESULTS AND CONCLUSIONS: Imprecision was lowest for VWF antigen and highest for collagen binding activity (CBA) with median coefficient of variation (CV) of 12% (interquartile range (IQR) 7%) and 23% (IQR 21%) respectively. Within the VWF PBA assays, the gain-of-function mutant GP1b binding (VWF: GP1bM) methods had the least imprecision (CV 9%, IQR 10%). All assays, including the various PBA methods had excellent consensus. The majority of laboratories agreed that normal (median consensus-82%, IQR 16%) and T1 VWD (median consensus-100%, IQR 9%) samples had normal multimer distribution. Consensus among laboratories for final interpretations was excellent for normal samples (median 81%, IQR 8%), good for T1 VWD samples (median 59%, IQR 9%), and fair for T2 VWD samples (median 44%, IQR 21%). Consensus on final interpretation decreased as sample complexity increased.

Performance of factor IX extended half-life product measurements in external quality control assessment programs.

BACKGROUND: Patients with hemophilia B are increasingly treated with extended half-life (EHL) factor IX (FIX) concentrates. For the laboratory, introduction of these EHL concentrates presents a major challenge. To understand the variation in FIX activity levels, all available diagnostic assays need to be directly compared. METHODS: The ECAT, UKNEQAS, and RCPAQAP have collaboratively performed a global survey to evaluate the quality of FIX measurements using FIX deficient plasma samples spiked with recombinant FIX (rFIX), rFIXFP, rFIXFc, and N9-GP to levels at typical FIX trough (6 IU/dL) and peak levels (60 IU/dL). Participants were asked to use their routine protocols, using one-stage assays (OSA) or chromogenic assays (CA). RESULTS: In samples spiked with 6 IU/dL product, median (25%-75% range) FIX activity levels (OSA), were 8.0 IU/dL (7.0-9.2) for rFIX, 6.0 IU/dL (4.0-7.1) for rFIXFP, 6.6 IU/dL (5.5-8.0) for rFIXFc, and 4.9 IU/dL (3.5-8.4) for N9-GP. In samples spiked with 60 IU/dL, FIX activity levels measured (using OSA) was 63.0 IU/dL (59.9-67.0) for rFIX, 42.5 IU/dL (28.2-47.0) for rFIXFP, 50.0 IU/dL (45.0-55.0) for rFIXFc, and 34.0 IU/dL (24.8-67.5) for N9-GP. Considerable differences were observed between reagents for all samples. With CA, there was also quite some variation, but no differences between reagents. CONCLUSION: Large variation is observed in the measurement of FIX activity levels after administration of rFIX and EHL FIX products. For N9-GP, most silica-based assays show especially high levels. It is essential to standardize and improve reliability of measurements of these concentrates as diagnosis and treatment monitoring is based on these results.

International external quality assessment for measurements of direct oral anticoagulants: results and recommendations.

There is limited information regarding the performance of tests for direct oral anticoagulants (DOACs). To generate more knowledge, the accuracy of DOAC tests were evaluated using external quality assessment data from multiple years. This data demonstrated a good correlation for the tests with a small overall interlaboratory variability (10% for dabigatran, rivaroxaban and apixaban and 12% for edoxaban). The greatest differences between the various reagents were observed for rivaroxaban, especially for concentrations below 100 ng/ml. In conclusion, the results show overall reliable DOAC levels with some differences between reagent groups. Important finding: clinical decision criteria could be affected by the choice of reagent.

Heparin-induced thrombocytopenia: An international assessment of the quality of laboratory testing.

BACKGROUND: Accurate diagnosis of heparin-induced thrombocytopenia (HIT) is essential to ensure timely treatment and prevent complications. Current diagnostic assays include enzyme-linked immunosorbent assays (ELISAs) and rapid immunoassays (RIs). RIs offer fast turnaround times but were not significantly represented in previous external proficiency testing challenges. OBJECTIVES: To use external proficiency testing to assess qualitative concordance for heparin/PF4 antibody detection. METHODS: From 2013 to 2017, the External Quality Control for Assays and Tests (ECAT) Foundation distributed 10 samples internationally. RESULTS: In total, 437 laboratories submitted 3149 results. ELISAs accounted for 1484 (47%) responses with RIs accounting for 1665 (53%) responses. RI use increased over the 5-year period. ELISAs classified 96% of both consensus positive and consensus negative samples concordantly. The coefficient of variation (CV) for positive sample optical densities (ODs) ranged from 35% to 50% when combining ELISA assay methods together. Quantitative RIs classified 97% of consensus-positive and 98% of consensus-negative samples concordantly. Qualitative RIs had a higher proportion of discordant responses and classified 88% of consensus-positive samples and 73% of consensus-negative samples concordantly. Of RIs only latex immunoassays and IgG specific chemiluminescent assays identified > 95% of samples concordantly with consensus. CONCLUSION: Quantitative RIs and ELISAs classify > 95% of samples concordantly. The ODs from different ELISA methods vary considerably and are not interchangeable. Qualitative RI use is increasing despite a greater proportion of discordant classifications. This includes a higher than expected number of negative classifications for consensus-positive samples among many RIs, challenging their use as "rule out" tests.

Analytical variation in factor VIII one-stage and chromogenic assays: Experiences from the ECAT external quality assessment programme.

BACKGROUND: Both one-stage (OSA) and chromogenic substrate assays (CSA) are used to measure factor VIII (FVIII) activity. Factors explaining analytical variation in FVIII activity levels are still to be completely elucidated. AIM: The aim of this study was to investigate and quantify the analytical variation in OSA and CSA. METHODS: Factors determining analytical variation were studied in sixteen lyophilized plasma samples (FVIII activity <0.01-1.94 IU/mL) and distributed by the ECAT surveys. To elucidate the causes of OSA variation, we exchanged deficient plasma between three company set-ups. RESULTS: On average, 206 (range 164-230) laboratories used the OSA to measure FVIII activity and 30 (range 12-51) used CSA. The coefficient of variation of OSA and CSA increased with lower FVIII levels (FVIII <0.05 IU/mL). This resulted in misclassification of a severe haemophilia A sample into a moderate or mild haemophilia A sample in 4/30 (13.3%) of CSA measurements, while this was 37/139 (26.6%) for OSA. OSA measurements performed with reagents and equipment from Werfen showed slightly lower FVIII activity (0.93, IQR 0.88-0.98 IU/mL) compared to measurements with Stago (1.07, IQR 1.02-1.14 IU/mL) and Siemens (1.03, IQR 0.97-1.07 IU/mL). Part of this difference is explained by the value of the calibrator. For CSA, the measured FVIII levels were similar using the different kits. CONCLUSIONS: In the lower range (<0.05 IU/mL), analytical variation of FVIII measurements is high in both OSA and CSA measurements. The variation in FVIII activity levels was partly explained by specific manufacturers. Further standardization of FVIII measurements and understanding of analytical variation is required.

Towards harmonization of external quality assessment/proficiency testing in hemostasis.

Quality in diagnostic testing represents a key target of laboratory medicine, for which an assurance around the quality of testing is expected from all involved in the process. Laboratories attempt to assure the quality of their testing by various processes, but especially by performance of internal quality control and external quality assessment (EQA). This is especially true for tests of hemostasis and coagulation. EQA in general provides information on test accuracy and on evaluation of long-term laboratory performance. EQA providers support laboratory performance by various means, including distribution of material for testing of analytes ("proficiency testing"), educational support through expert advice, distribution of publications or case series. Participation in EQA is often a laboratory accreditation requirement. This review aims to identify some of the strengths and weaknesses of EQA, and targets attempts towards harmonization of EQA practice, in order to achieve the best outcome for participant laboratories and, thus, for patients and their clinical care providers.

Lack of grading agreement among international hemostasis external quality assessment programs.

: Laboratory quality programs rely on internal quality control and external quality assessment (EQA). EQA programs provide unknown specimens for the laboratory to test. The laboratory's result is compared with other (peer) laboratories performing the same test. EQA programs assign target values using a variety of methods statistical tools and performance assessment of 'pass' or 'fail' is made. EQA provider members of the international organization, external quality assurance in thrombosis and hemostasis, took part in a study to compare outcome of performance analysis using the same data set of laboratory results. Eleven EQA organizations using eight different analytical approaches participated. Data for a normal and prolonged activated partial thromboplastin time (aPTT) and a normal and reduced factor VIII (FVIII) from 218 laboratories were sent to the EQA providers who analyzed the data set using their method of evaluation for aPTT and FVIII, determining the performance for each laboratory record in the data set. Providers also summarized their statistical approach to assignment of target values and laboratory performance. Each laboratory record in the data set was graded pass/fail by all EQA providers for each of the four analytes. There was a lack of agreement of pass/fail grading among EQA programs. Discordance in the grading was 17.9 and 11% of normal and prolonged aPTT results, respectively, and 20.2 and 17.4% of normal and reduced FVIII results, respectively. All EQA programs in this study employed statistical methods compliant with the International Standardization Organization (ISO), ISO 13528, yet the evaluation of laboratory results for all four analytes showed remarkable grading discordance.

External quality assessment of platelet disorder investigations: results of international surveys on diagnostic tests for dense granule deficiency and platelet aggregometry interpretation.

The quality of platelet aggregation and dense granule deficiency testing is important for diagnosing platelet function disorders. After a successful pilot exercise on diagnosing platelet dense granule deficiency by electron microscopy (EM), the North American Specialized Coagulation Laboratory Association (NASCOLA) has launched regular external quality assurance (EQA) for dense granule EM, as well as for the interpretation of platelet aggregation findings. EQA records were analyzed to assess performance. For EM EQA, between 2009 and 2011, there was excellent performance in distinguishing normal from dense granule-deficient samples and good (>70%) agreement on classifying most electron dense structures in platelets. For aggregation EQA, some normal variants were misclassified and overall case interpretations were more acceptable for rare disorders than for common findings. NASCOLA experiences with these EQAs indicate that there is a need to improve the quality of platelet disorder evaluations. For aggregometry interpretations, deficits in performance could be addressed by translating guideline recommendations into practice.

External quality assessment of point-of-care International Normalized Ratio (INR) testing in Europe.

BACKGROUND: Point-of-care testing (POCT) of prothrombin time, expressed as International Normalized Ratio (INR), is widely used to monitor patients in oral anticoagulation treatment. Guidelines recommend that POCT users should participate in an external quality assessment (EQA) scheme whenever available. The aim of this study was to investigate which European countries provide EQA for POCT INR and to compare how these schemes are organized. METHODS: Thirty European countries were invited to participate in this study. Those who reported that they provide EQA for POCT INR filled in a questionnaire dealing with different aspects of their schemes. RESULTS: Nineteen countries reported that they do not provide EQA for POCT INR, while 12 organizations from nine countries reported that they provide this service. Most of these countries circulate lyophilized samples with for the participants unknown target values. Samples with certified INR values and procedures using split samples with fresh patient samples are also used. The acceptability limits vary from 15% to 30%, and the total number of samples circulated per year varies from 1 to 12. Most of the countries organize educational activities together with their schemes. CONCLUSIONS: This study demonstrates that there is a wide variation in the way EQA for POCT INR is performed in Europe and that there are many European countries that do not provide this service. Even though our findings indicate that EQA for POCT INR draws some challenges, especially in providing suitable control materials, participation in such schemes is considered useful.

External quality assessment for thrombin generation tests: an exploration.

External Quality Control of Diagnostic Assays and Tests (ECAT) surveys on thrombin generation rests (TGTs) have identified that various tests show a more than 30-fold difference in time to peak (TTP). The survey included pooled normal plasmas, microparticle (MP)-depleted plasmas, and factor (F)XII-deficient patient plasma. Between 4 and 11 laboratories participated per test; analyzed were a time (TTP) and quantity variable (AUC). MP depletion of plasma showed a progressive increase in TTP up to 29% with a decreasing tissue factor. The same was found for the AUC with the largest decrease of 38%. It was observed that MP depletion showed large individual differences. The FXII-deficient plasma showed no effect on TTP for rapid tests, but for slow tests it increased from 248 to 331%. The AUC declined gradually the slower the test, reaching a decline of 85%. The effects of FXII deficiency were not mimicked by the addition of corn trypsin inhibitor but were confirmed by inhibiting activated FXI. Interlaboratory variability was between 11% and 57% for all methods, showing differences mainly related to the used normal pooled plasma. The different sensitivities of TGTs to MPs and contact activation predict that they will associate differently with clinical situations, according to which of these aspects are important. Future ECAT surveys should include samples with variation in MPs and contact activation to match with features of the TGT variants.

Laboratory assessment of factor VIII inhibitor titer: the North American Specialized Coagulation Laboratory Association experience.

Quantification of inhibitory antibodies against infused factor VIII (FVIII) has an important role in the management of patients with hemophilia A. This article summarizes results from the largest North American FVIII inhibitor proficiency testing challenge conducted to date. Test samples, 4 negative and 4 positive (1-3 Bethesda units [BU]/mL), were distributed by the ECAT Foundation in conjunction with the North American Specialized Coagulation Laboratory Association and analyzed by 38 to 42 laboratories in 2006 and 2007. Whereas laboratories were able to distinguish between the absence and presence of low-titer FVIII inhibitors, the intralaboratory coefficient of variation was high (30%-42%) for inhibitor-positive samples, and the definition of lower detection limits of the assay was variable (0-1 BU/mL). Most laboratories performed the Bethesda assay with commercially supplied buffered normal pooled plasma in a 1:1 mix with patient plasma. These data provide information for the development of consensus guidelines to improve FVIII inhibitor quantification.

The between-laboratory variation of factor VIII inhibitor testing: the experience of the external quality assessment program of the ECAT foundation.

The detection and quantification of factor VIII (FVIII) inhibitors is clinically important both for the identification of hemophilia A patients with inhibitors and for the management of immune tolerance treatment. Only limited data are available on the between-laboratory variation of FVIII inhibitor testing. This report describes the evaluation of the results of the large-scale external quality assessment program of the European Concerted Action on Thrombosis Foundation. This study includes the results of six different surveys for the period 2006 to 2008 with 100 to 170 participating laboratories. The overall between-laboratory variation ranged from 28% to 52% with a slightly lower variation for the Nijmegen assay (approximately 39%) on average than for the Bethesda assay (approximately 45%). The use of buffered normal pooled plasma as FVIII source showed better performance compared with the use of nonbuffered pooled plasma; likewise the use of FVIII-deficient plasma compared with the use of imidazole buffer. However, the combination of both was essential for lowest between-laboratory variation. The Nijmegen assay also showed better performance with respect to specificity and sensitivity than the Bethesda assay, although the results for neither were entirely satisfactory. In general, it can be concluded that the measurement of FVIII inhibitory antibodies with the Nijmegen assay should be favored over the use of the Bethesda assay. However, further improvement of the laboratory test for FVIII inhibitors is urgently needed.

A national field study of quality assessment of CoaguChek point-of-care testing prothrombin time monitors.

A system for quality assessment (QA) of the CoaguChek (Roche Diagnostics, Mannheim, Germany) point-of-care testing prothrombin time monitor has been developed by the European Concerted Action on Anticoagulation. Hitherto there has not been an adequate rapid method for CoaguChek QA. Sets of 5 certified international normalized ratio (INR) plasma samples were tested on 539 CoaguChek monitors by experienced staff at 9 Netherlands Thrombosis Centers and results compared with certified INR. A 15% or more deviation has been classified as significant deviation. Overall mean and certified INR values were similar, but 20.3% of participants showed a 15% or more deviation from the certified INR on at least 1 of the 5 QA plasma samples. Statistically significant differences in results with different lots of CoaguChek test strips were found. There is need for large scale QA of CoaguChek monitors. The importance of the 5 CoaguChek certified INR QA plasma samples being tested on a single occasion is demonstrated.

An external quality assessment program for von Willebrand factor laboratory analysis: an overview from the European concerted action on thrombosis and disabilities foundation.

The laboratory diagnosis of von Willebrand disease (vWD) is complex and requires a panel of different laboratory tests. Because of this complexity, a proper quality control process is necessary. Since 2003, the European Concerted Action on Thrombosis and Disabilities Foundation has provided an external quality control program for several laboratory tests included in the diagnosis of vWD. Currently, ~180 different laboratories participate in this program, of which the vast majority perform both von Willebrand factor (vWF):antigen (Ag) and activity tests. The lowest between-laboratory variation was observed for the vWF antigen assay (10 to 24%), with a better performance for the latex immunoassay (8 to 24%) than the enzyme immunoassay (13 to 25%). Both the ristocetin cofactor activity assay (RCo) and the collagen-binding assay showed a higher between-laboratory variation (20 to 40% and 17 to 29%, respectively). We have observed that the within-laboratory repeatability for normal samples ranged from 0 to 40% for the antigen assay and from 0 to 86% for the ristocetin cofactor activity assay. Normal samples were interpreted correctly by the majority of the participants. However, type 1 vWD samples were wrongly interpreted by 20 to 40% of the participants, which was mainly caused by a discordance in the vWF:RCo/vWF:Ag ratio. It can be concluded that further improvement in the laboratory diagnosis of vWD is necessary.

External quality assessment and the laboratory diagnosis of thrombophilia.

External quality assessment is a tool to compare the result of a particular laboratory test in relation to those of other laboratories as well as to assess the performance of a laboratory test over a prolonged period of time. We evaluated the relationship between the between-laboratory variation and the sample category (normal, borderline, and abnormal) for antithrombin, protein C, protein S, and the activated protein C resistance test. Only for antithrombin and protein S was a significant relationship (0.004 < p < 0.012) observed. The effect of the between-laboratory variation of the different sample categories on the clinical interpretation was investigated. With the exception of free protein S antigen, all variables showed a significant relationship (0.004 < p < 0.045) between the sample category and the percentage of misclassification. Because in clinical practice a stable test performance over a prolonged period of time is important, we evaluated the quality of test performance using the long-term analytical coefficient of variation (LCVa). A wide range in the LCVa was observed for antithrombin, protein C, and protein S. Less than half of the participants could fulfill the quality specification for diagnostic testing (LCVa < or = 0.58 x total biological variation). This study shows that a more stable performance of laboratory tests involved in the screening of thrombophilia over a prolonged period of time is necessary.

Long-term analytical performance of hemostasis field methods as assessed by evaluation of the results of an external quality assessment program for antithrombin.

BACKGROUND: It is important for a laboratory to know the stability of performance of laboratory tests over time. The aim of this study was to adapt from the field of clinical chemistry a method to assess the long-term analytical performance of hemostasis field methods. METHODS: The linear regression model was used to compare the laboratory results with the consensus mean value of a survey. This model was applied to plasma antithrombin activity using the data for 82 laboratories, collected between 1996 and 1999 in the European Concerted Action on Thrombosis (ECAT) external quality assessment program. The long-term total, random, and systematic error were calculated. The variables introduced to define the long-term performance in this model were the long-term analytical CV (LCV(a)) and the analytical critical difference (ACD), which indicates the minimum difference necessary between two samples measured on a long-term time-scale to consider them statistically significantly different. RESULTS: The systematic error (bias) ranged from 4.5 to 103 units/L. The random error ranged from 24.4 to 242 units/L. For the majority of the laboratories, random error was the main component (>75%) of the total error. The LCV(a), after adjustment for the contribution of the bias, ranged from 2.8% to 48%. The ACD ranged from 78 to 1290 units/L with a median value of 190 units/L. No statistically significant differences were observed for either LCV(a) or ACD between the two different measurement principles for antithrombin activity based on the inhibition of either thrombin or factor Xa. CONCLUSIONS: This linear regression model is useful for assessing the total error, random error, and bias for hemostasis field methods. The LCV(a) and ACD for measurement on a long-term time-scale appear to be useful for assessing the long-term analytical performance.