Expressing analytical performance from multi-sample evaluation in laboratory EQA.

BACKGROUND: To provide its participants with an external quality assessment system (EQAS) that can be used to check trueness, the Dutch EQAS organizer, Organization for Quality Assessment of Laboratory Diagnostics (SKML), has innovated its general chemistry scheme over the last decade by introducing fresh frozen commutable samples whose values were assigned by Joint Committee for Traceability in Laboratory Medicine (JCTLM)-listed reference laboratories using reference methods where possible. Here we present some important innovations in our feedback reports that allow participants to judge whether their trueness and imprecision meet predefined analytical performance specifications. METHODS: Sigma metrics are used to calculate performance indicators named 'sigma values'. Tolerance intervals are based on both Total Error allowable (TEa) according to biological variation data and state of the art (SA) in line with the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Milan consensus. RESULTS: The existing SKML feedback reports that express trueness as the agreement between the regression line through the results of the last 12 months and the values obtained from reference laboratories and calculate imprecision from the residuals of the regression line are now enriched with sigma values calculated from the degree to which the combination of trueness and imprecision are within tolerance limits. The information and its conclusion to a simple two-point scoring system are also graphically represented in addition to the existing difference plot. CONCLUSIONS: By adding sigma metrics-based performance evaluation in relation to both TEa and SA tolerance intervals to its EQAS schemes, SKML provides its participants with a powerful and actionable check on accuracy.

Commutability of proficiency testing material containing tobramycin: a study within the framework of the Dutch Calibration 2.000 project.

BACKGROUND: Results from external quality assessment schemes (EQASs) can provide information about accuracy and comparability of different measurement methods, provided that the material used in these schemes behave identical to patient samples among the different methods, a characteristic also known as commutability. The aim of this study was to assess the commutability of different matrices for the material used in an EQAS for tobramycin. METHODS: Proficiency testing material (PTM) and patient samples containing tobramycin were prepared, collected, pooled, and distributed to participating laboratories for analysis. Low, medium, and high tobramycin concentrations in liquid human, liquid bovine and lyophilized bovine serum were tested in this study. The patient serum results of every laboratory were plotted against each of the other laboratories, and the distances of the PTM results to the patient serum regression line were calculated. For comparison, these distances were divided by the average within-laboratory standard deviation (SDwl) of the results reported in the official EQAS for tobramycin, resulting in a relative residual. The commutability decision limit was set at 3 SDwl. RESULTS: With 10 laboratories participating in this study, 45 laboratory couples were formed. For human serum, only one relative residual for high concentrations of tobramycin was found outside the commutability decision limit. For liquid and lyophilized bovine sera, the number of relative residuals outside the decision limit was between 15 and 18 for low, medium, and high tobramycin concentrations. CONCLUSIONS: The PTM used for tobramycin is preferably prepared with human serum.

A Multilaboratory Commutability Evaluation of Proficiency Testing Material for Carbamazepine and Valproic Acid: A Study Within the Framework of the Dutch Calibration 2000 Project.

BACKGROUND: Medical laboratories are required to participate in interlaboratory comparisons of the analyses they perform. The materials used in these comparisons need to be of sufficient quality so that the comparison provides a picture of the performances. One of the main characteristics of the testing material is commutability, which is the ability of a material to yield the same numerical relationships between results of measurements as those relationships obtained when the same procedures are applied to patient samples. The aim of this study was to assess the commutability of 3 different matrices for the preparation of proficiency testing material (PTM) for the analysis of carbamazepine and valproic acid. METHODS: Patient samples and PTM containing various concentrations of carbamazepine and valproic acid were collected, prepared, and shipped to different laboratories for analysis. Reported results for patient samples from each laboratory were plotted against results for patient samples of each of the other laboratories, and the corresponding regression line was calculated. The distance of results from PTM to the regression line is a measure for commutability. The distance is expressed as a multiple of the SDwl (average within-laboratory SD as calculated from external quality assessment scheme results) and referred to as relative residual. A commutability decision limit of 2 SDwl was set. RESULTS: For carbamazepine and valproic acid, a total of 78 and 105 laboratory couples respectively could be formed. The number of relative residuals for liquid human serum outside the commutability decision limit was 1, 4, and 0 for low, medium, and high concentrations of carbamazepine, respectively and 3, 1, and 0 for low, medium, and high concentrations of valproic acid, respectively. In both liquid and lyophilized bovine sera, the number of relative residuals outside the commutability decision limit was between 2 and 15 and between 6 and 21 for carbamazepine and valproic acid, respectively. CONCLUSIONS: Although not all results for PTM with carbamazepine and valproic acid are within the commutability decision limits, a preference for human serum can be seen.

Defining acceptable limits for the metrological traceability of specific measurands.

Although manufacturers are compelled by the European IVD Directive, 98/79/EC, to have traceability of the values assigned to their calibrators if suitable higher order reference materials and/or procedures are available, there is still no equivalence of results for many measurands determined in clinical laboratories. The adoption of assays with metrological traceable results will have a significant impact on laboratory medicine in that results will be equivalent across different laboratories and different analytical platforms. The IFCC WG on Allowable Errors for Traceable Results has been formed to define acceptable limits for metrological traceability chains for specific measurands in order to promote the equivalence of patient results. These limits are being developed based on biological variation for the specific measurands. Preliminary investigations have shown that for some measurands, it is possible for manufacturers to assign values to assay calibrators with a measurement uncertainty that allows the laboratory enough combined uncertainty for their routine measurements. However, for other measurands, e.g., plasma sodium, current assays are too imprecise to fulfil limits based on biological variation. Although an alternative approach based on probability theory is being investigated, the most desirable approach would be for industry to improve measurement methods so that they meet clinical requirements.

Second round robin for plasma hepcidin methods: first steps toward harmonization.

Measurements of the iron regulatory hormone hepcidin by various methodologies and laboratories are not harmonized. As a result different numeric results are obtained for the same clinical sample. We investigated whether better agreement between plasma hepcidin methods can be achieved by harmonization. Native plasma pools (n = 11) of a variety of hepcidin concentrations and blank plasma spiked with three different quantities of synthetic hepcidin-25 purchased from two different commercial sources (n = 6), were distributed in duplicate among 21 methods worldwide. We assessed commutability by comparing results from synthetic hepcidin with those from native samples in various method couples by Bland-Altman plots. Methods differed substantially in absolute values and reproducibility. For the majority of methods we found that samples with synthetic hepcidin-25 were noncommutable with the native samples. In an attempt to harmonize by using native hepcidin results, we selected two methods that showed good mutual agreement of native results and calculated consensus values as the medians for the 11 duplicate native samples obtained by these two methods. Finally, we constructed algorithms enabling the laboratories to calculate the hepcidin consensus (HEPCON) value using their own native hepcidin results. We found that the use of these algorithms substantially reduced the between-method CV. Until commutable materials are defined, hepcidin harmonization can be achieved by exploiting specific algorithms, allowing each lab to report their native hepcidin concentrations in HEPCON values. This study represents the first step toward harmonization of plasma hepcidin methods and facilitates aggregation of hepcidin data from different research investigations.

SKML-Quality Mark for point-of-care test (POCT) glucose meters and glucose meters for home-use.

BACKGROUND: Point-of-care glucose meters are used increasingly in semi- and non-professional context. The quality of glucose measurements depends on the quality of the equipment, the quality of use, and the pre-analytical conditions. In this article, a External Quality Assessment Scheme (SKML)-Quality Mark for point-of-care test (POCT) and self-test glucose meters is proposed, assessing analytical quality and technical quality. The analytical requirements are based on the biological variation concept, and a system to assess meters for the SKML-Quality Mark is described. Using the proposed system as an example, 14 meters were tested. METHODS: The analytical quality of the POCT and self-test equipment was assessed for plasma calibrated glucose values by comparison with a trueness verified method traceable to the IFCC reference method in an accredited clinical laboratory. The concept is based on the biological variation system. The SKML-Quality Mark comprises the following criteria for blood glucose equipment: 1) Fulfilment of compliance with ISO 15197 and/or TNO guideline criterion; 2) Fulfilment of the total allowable error (TAE) criterion; 3) Fulfilment of the total allowable linearity bias criterion; 4) Fulfilment of the total allowable interfering substances bias criterion; and 5) Fulfilment of the haematocrit criterion. RESULTS: The proposed SKML-Quality Mark system was tested on 14 commercial home-use meters. The TAE criterion is violated by two meters. The main reason for the violation is bias. For the majority of meters, the Passing and Bablok regression confidence interval does not include the intercept of 0.0 and slope of 1.0. In addition, Syx indicates dispersion around the line or non-linearity. The bias and total error at three different concentrations were investigated as part of the quality mark, resulting in disapproval of the Dicomed Sensocard Plus meter. The bias was significant for the Wellion Linus. With respect to interfering substances, bias of the same magnitude and sign as the bias without additive was seen for all meters for acetaminophen, indicating no additional interference. For ascorbic acid, an additional bias was seen for several meters. However, significant bias was demonstrated for the Sensocard Plus and Glucocard X-meter. CONCLUSIONS: The biological variation concept offers a scientific basis for assessment of acceptable deviation. The concept is extended in the SKML-Quality Mark correcting for the limited number of measurements that can be performed while assessing home-use or POCT meters. The results show that three out of 14 meters fail the proposed quality mark.

Commutability of proficiency testing material containing amitriptyline and nortriptyline: A study within the framework of the Dutch Calibration 2.000 project.

BACKGROUND: External quality assessment schemes (EQAS) can provide important information regarding accuracy and comparability of different measurement methods if the sample matrices are composed of commutable material. The aim of this study was to assess the commutability of different matrices for the material used in an EQAS for amitriptyline and nortriptyline. METHODS: Proficiency testing material (PTM) and patient samples containing amitriptyline and nortriptyline were prepared, collected, pooled, and distributed to participating laboratories for analysis. Low, medium and high concentrations of both drugs in liquid pooled human, lyophilized human and lyophilized bovine serum were tested in this study. The measurement deviation of the PTM results to the patient serum regression line were normalized by dividing trough the average within-laboratory SD (SDwl) derived from the results reported in the official EQAS, resulting in a relative residual. The commutability decision limit was set at 3 SDwl. RESULTS: With 10 laboratories participating in this study, 45 laboratory couples were formed. All matrix types delivered several relative residuals outside the commutability decision limit. The number and the magnitude of relative residuals for both drugs were lower for liquid human sera as compared to lyophilized human and bovine sera. CONCLUSIONS: The PTM used for amitriptyline and nortriptyline is preferably prepared with human serum, although not all relative residuals are within the commutability decision limit.

The EC4 register of European clinical chemists and EC4 activities.

The freedom of movement of people and goods within the European Union (EU) has a large impact for the member states. Particularly within health care it is important to recognize, or if necessary obtain, an adequate level of the quality of profession and practice, so that citizens know that health care is offered in their country at a level comparable to other countries. The importance of recognition also applies to laboratory medicine. European Communities Confederation of Clinical Chemistry (EC4) is the organization of societies for clinical chemistry and laboratory medicine in the EU. In Europe, health care develops in the direction where patients are treated in a health care chain environment. In this chain, patients move quickly from primary health institutes to secondary and tertiary institutes, and vice versa. This situation involves many health care workers including several laboratories. Diagnosis and therapy are now 'core business' of health care. Medical laboratories play an essential role in this. The broad spectrum of medical laboratory investigations make consultancy of medical laboratory specialists ever more important. The quality of both professionals and laboratories, as well as continuity of laboratory data within and between laboratories, are of utmost importance.EC4 is active in giving support to attain such quality. In most countries, this is the case at present. EC4 plays a central role in the Coordination of Automatic Recognition of Equivalence of Standards (CARE), if such a level exists or is achieved. Such CARE is focussed at three levels, the profession, quality of laboratories and calibration of laboratory data. The EC4 Register of European Clinical Chemists is open for colleagues educated in (bio)chemistry, pharmacy, biology as well as medicine, and trained according to the EC4 Syllabus. Equivalence of standards has been granted to national training schemes of 13 European Union countries. Since its opening in 1998, the number of applicants is growing steadily and quickly, reaching 1225 in May 2001.EC4 has published essential criteria for quality systems of medical laboratories, which formed the basis for a ISO draft international standard regarding quality and competence of medical laboratories.EC4 stimulates projects like the Calibration 2000 project in the Netherlands which focus on continuity of laboratory data, within-as well as between-laboratories.

Accreditation of medical laboratories in the European Union.

BACKGROUND: Using a questionnaire, the EC4 (European Communities Confederation of Clinical Chemistry and Laboratory Medicine) has collated an inventory of the accreditation procedures for medical laboratories in the EU. RESULTS AND DISCUSSION: Accreditation of medical laboratories in the countries of the EU is mostly carried out in cooperation with national accreditation bodies. These national accreditation bodies work together in a regional cooperation, the European Cooperation for Accreditation (EA). Professionals are trained to become assessors and play a prominent role in the accreditation process. The extent of the training is diverse, but assessors are kept informed and up-to-date by annual meetings. The frequency of assessments and surveillance visits differs from country to country and ranges from 1 to 4 years. More harmonisation is needed in this respect, based on a frequency that can be pragmatically handled by laboratory professionals. In the majority of EA bodies, accreditation is carried out on a test-by-test basis. Many professionals would prefer accreditation of the entire service provided within the actual field of testing (i.e., haematology, immunology, etc.), with accreditation granted if the majority of tests offered within a service field fulfil the requirements of the ISO 15189 standard. The scope of accreditation is a major point of discussions between the EC4 Working Group on Accreditation and representatives of accreditation bodies in the EA Medical Laboratory Committee.

Internal quality control system for non-stationary, non-ergodic analytical processes based upon exponentially weighted estimation of process means and process standard deviation.

The analytical processes in clinical laboratories should be considered to be non-stationary, non-ergodic and probably non-stochastic processes. Both the process mean and the process standard deviation vary. The variation can be different at different levels of concentration. This behavior is shown in five examples of different analytical systems: alkaline phosphatase on the Hitachi 911 analyzer (Roche), vitamin B12 on the Access analyzer (Beckman), prothrombin time and activated partial thromboplastin time on the STA Compact analyzer (Roche) and PO2 on the ABL 520 analyzer (Radiometer). A model is proposed to assess the status of a process. An exponentially weighted moving average and standard deviation was used to estimate process mean and standard deviation. Process means were estimated overall and for each control level. The process standard deviation was estimated in terms of within-run standard deviation. Limits were defined in accordance with state of the art- or biological variance-derived cut-offs. The examples given are real, not simulated, data. Individual control sample results were normalized to a target value and target standard deviation. The normalized values were used in the exponentially weighted algorithm. The weighting factor was based on a process time constant, which was estimated from the period between two calibration or maintenance procedures. The proposed system was compared with Westgard rules. The Westgard rules perform well, despite the underlying presumption of ergodicity. This is mainly caused by the introduction of the starting rule of 12s, which proves essential to prevent a large number of rule violations. The probability of reporting a test result with an analytical error that exceeds the total allowable error was calculated for the proposed system as well as for the Westgard rules. The proposed method performed better. The proposed algorithm was implemented in a computer program running on computers to which the analyzers were linked on-line. Each result was evaluated on-line, and a limit violation was immediately reported. The system has performed satisfactorily in our laboratory for ten analyzers for over 1 year.