The Empower project - a new way of assessing and monitoring test comparability and stability.

BACKGROUND: Manufacturers and laboratories might benefit from using a modern integrated tool for quality management/assurance. The tool should not be confounded by commutability issues and focus on the intrinsic analytical quality and comparability of assays as performed in routine laboratories. In addition, it should enable monitoring of long-term stability of performance, with the possibility to quasi "real-time" remedial action. Therefore, we developed the "Empower" project. METHODS: The project comprises four pillars: (i) master comparisons with panels of frozen single-donation samples, (ii) monitoring of patient percentiles and (iii) internal quality control data, and (iv) conceptual and statistical education about analytical quality. In the pillars described here (i and ii), state-of-the-art as well as biologically derived specifications are used. RESULTS: In the 2014 master comparisons survey, 125 laboratories forming 8 peer groups participated. It showed not only good intrinsic analytical quality of assays but also assay biases/non-comparability. Although laboratory performance was mostly satisfactory, sometimes huge between-laboratory differences were observed. In patient percentile monitoring, currently, 100 laboratories participate with 182 devices. Particularly, laboratories with a high daily throughput and low patient population variation show a stable moving median in time with good between-instrument concordance. Shifts/drifts due to lot changes are sometimes revealed. There is evidence that outpatient medians mirror the calibration set-points shown in the master comparisons. CONCLUSIONS: The Empower project gives manufacturers and laboratories a realistic view on assay quality/comparability as well as stability of performance and/or the reasons for increased variation. Therefore, it is a modern tool for quality management/assurance toward improved patient care.

Measurements for 8 common analytes in native sera identify inadequate standardization among 6 routine laboratory assays.

BACKGROUND: External quality assessment (EQA) with commutable samples is essential for assessing the quality of assays performed by laboratories, particularly when the emphasis is on their standardization status and interchangeability of results. METHODS: We used a panel of 20 fresh-frozen single-donation serum samples to assess assays for the measurement of creatinine, glucose, phosphate, uric acid, total cholesterol, HDL cholesterol, LDL cholesterol, and triglycerides. The commercial random access platforms included: Abbott Architect, Beckman Coulter AU, Ortho Vitros, Roche Cobas, Siemens Advia, and Thermo Scientific Konelab. The assessment was done at the peer group level and by comparison against the all-method trimmed mean or reference method values, where available. The considered quality indicators were intraassay imprecision, combined imprecision (including sample-matrix interference), bias, and total error. Fail/pass decisions were based on limits reflecting state-of-the-art performance, but also limits related to biological variation. RESULTS: Most assays showed excellent peer performance attributes, except for HDL- and LDL cholesterol. Cases in which individual assays had biases exceeding the used limits were the Siemens Advia creatinine (-4.2%), Ortho Vitros phosphate (8.9%), Beckman Coulter AU triglycerides (5.4%), and Thermo Scientific Konelab uric acid (6.4%), which lead to considerable interassay discrepancies. Additionally, large laboratory effects were observed that caused interlaboratory differences of >30%. CONCLUSIONS: The design of the EQA study was well suited for monitoring different quality attributes of assays performed in daily laboratory practice. There is a need for improvement, even for simple clinical chemistry analytes. In particular, the interchangeability of results remains jeopardized both by assay standardization issues and individual laboratory effects.

A statistical basis for harmonization of thyroid stimulating hormone immunoassays using a robust factor analysis model.

BACKGROUND: Between-method equivalence ideally is achieved by calibration against an SI-traceable reference measurement procedure. For measurement of thyroid stimulating hormone (TSH), it is unlikely to accomplish this goal in mid-term. Therefore, we investigated a statistical alternative based on a factor analysis (FA) model. METHODS: The FA model was applied to TSH results for 94 samples generated by 14 immunoassays (concentration range: 0.0005-78 mIU/L). The dataset did not fulfill the assumption of a homogeneous sample from an elliptically symmetric distribution, and, therefore, required standardization prior to application of the FA model. As outliers and missing values also occurred, the key quantities of the FA model had to be estimated with a method that can handle these complications. We selected a robust alternating regressions (RAR) method, which replaces in the minimization criterion of the fitting process the squared differences between results xij and model fit x^ij ${\hat x_{ij}}$ by a weighted absolute difference. The weights are adaptively determined in successive regressions, which down weighs the outliers. The weights for missing values are set to zero. RESULTS: The quality of the estimated targets was reflected by their central position in the distributions, and description of the relationship between results and targets by a simple two-parameter regression equation with high correlation coefficients and low SDs of the percentage-residuals. Mathematical recalibration eliminated the method differences and improved the between-method CV from 11% to 6%. CONCLUSIONS: RAR applied to a multimethod comparison dataset hampered by outliers and missing values, is fit to the purpose of harmonization.

Status of serum-calcium and -albumin measurement in Argentina assessed in 300 representative laboratories with 20 fresh frozen single donation sera.

BACKGROUND: The Fundación Bioquímica Argentina (FBA) performs external quality assessment (EQA) of >3200 laboratories. However, FBA realizes that sample non-commutability and predominant use of heterogeneous systems may bias the estimated performance and standardization status. To eliminate these confounding factors, a study using frozen single donation sera was undertaken with the focus on serum-calcium and -albumin measurement. METHODS: Target values were established from the results produced with homogeneous systems. In groups of n=7, system effects were investigated. Laboratory performance was evaluated from the correlation coefficient r between the measurement results for all sera and the target values. This allowed ranking of the laboratories and judgment of the deviation for individual samples (total error) against a 10% limit. The total error specification was a deviation for ≥ 5 samples exceeding 10% and/or causing a result outside the laboratory's reference interval. RESULTS: For calcium (n=303) (range: 2.06-2.42 mmol/L), 81 laboratories had an r-value <0.6, 43 even <0.4; the total error was relevant for 97 (10% limit) and 111 (reference interval) laboratories. For albumin (n=311) (range: 34.7-45.7 g/L) r was <0.7 (<0.4) in 44 (16) laboratories; 83 and 36 laboratories exceeded the total error criteria. Laboratories using homogeneous systems were generally ranked higher by correlation. System effects were moderate for calcium, but significant for albumin. CONCLUSIONS: The study demonstrated the need to improve the quality and harmonization of calcium and albumin testing in the investigated laboratories. To achieve this objective, we promote co-operation between laboratories, EQA provider and manufacturers.

Full-scan mass spectral evidence for 3-epi-25-hydroxyvitamin D3 in serum of infants and adults.

BACKGROUND: Since the introduction of liquid chromatography-mass spectrometry (LC/MS) for assessing vitamin D status, it has been shown that the C-3 epimer can account for a significant proportion of circulating 25-hydroxyvitamin D3 (25OHD3) concentrations in infants. However, some question whether monitoring a single MS transition at a chromatographic retention time typical for 3-epi-25OHD3 sufficiently warrants conclusions about the identity of the substance generating the signal. Therefore, we aimed to substantiate the evidence for 3-epi-25OHD3 in infants by collision induced dissociation (CID)-MS/MS product ion scans. A second objective was mass spectrometric investigation of the presence and prevalence of the 3-epi metabolite in serum from adults. METHODS: Serum samples from six infants and 32 adults were studied using an ultra performance LC/tandem MS (UPLC/MS/MS) method designed to separate the 3-epi-25OHD3 from 25OHD3. Samples were submitted to liquid/liquid extraction and Sephadex LH-20 fractionation, prior to column-switching UPLC with MS/MS recording of CID product ion spectra of the [M+H](+) precursor ion. The respective standards were analyzed under identical UPLC/MS/MS conditions for comparison. RESULTS: In the chromatograms of all samples, two peaks eluted with retention characteristics and spectra closely matching those observed for the 25OHD3 and the 3-epi standards. The percentage of the 3-epi metabolite relative to 25OHD3 in infants ranged from 15% to 41%, and in adults from 2.5% to 17%. CONCLUSIONS: This preliminary finding suggests that the prevalence of 3-epi-25OHD3 in serum of infants is considerable, and that even in adults the concentrations of this form should not be neglected.

Specifications for trueness and precision of a reference measurement system for serum/plasma 25-hydroxyvitamin D analysis.

BACKGROUND: The divergence in analytical quality of serum/plasma 25-hydroxy-vitamin D analysis calls for defining specifications for a reference measurement system. METHODS: Fundamentally, in a reference measurement system, there should be a relationship between the analytical specifications for higher- (reference) and lower-order (routine) measurements. Therefore, when setting specifications, we started with limits for routine imprecision (CV(rou)) and bias (B(rou)) using 4 models: (1) the misclassifications in diagnosis, (2) biological variation data (reference interval (RI) and monitoring), (3) expert recommendations, and (4) state-of-the-art performance. Then, we used the derived goals to tailor those for reference measurements and certified reference materials (CRMs) for calibration by setting the limits for CV(ref) at 0.5 CV(rou), B(ref) at 0.33 B(rou)(,) max. uncertainty (U(max)) at 0.33 B(ref). RESULTS: The established specifications ranged between CV(rou)

Method validation across the disciplines--critical investigation of major validation criteria and associated experimental protocols.

Analytical method development should aim at delivering reliable measurements within a given application. This implies that method validation is integrated in the development process, because it enables to establish a method's performance capabilities, and to demonstrate its fitness-for-purpose. Although analytical chemists mostly are familiar with the validation guidelines within the discipline of their responsibility, we believe that they may take advantage of a better acquaintance with recommendations among disciplines. Therefore, we review the guidance given in 4 disciplines (laboratory medicine, pharmacy, environment, and food), with emphasis on the proposed experimental protocols, acceptance criteria and interpretation strategies by statistical significance testing. Last but not least, we give incentive towards a modernized validation design.

About the z-multiplier in total error calculations.

BACKGROUND: Total error (TE) in analytical measurement is calculated as systematic error (SE) plus z-times random error (RE). The z-multiplier is typically chosen at the 95% probability level, being 1.96 in the absence of SE is of considerable magnitude (one-sided 95% probability). Up to now, no SE/RE ratio dependent z-values have been considered. Here, we present z-values for SE/RE ratios ranging from 0 to 1. METHODS: The z-multiplier (95% probability level) was empirically obtained by modulation of the standard normal distribution with the EXCEL NORMDIST function (Microsoft EXCEL 2002). In total, five distributions representing SE/RE ratios between 0 and 1 were calculated, so that the total probability outside a TE boundary +/- 1.96 sigma was in the order of 5.013%. RESULTS: For distributions with SE/RE ratios ranging from 0 to 1 that satisfy the aforementioned total probability outside a TE boundary +/- 1.96 sigma, we found that the associated z-multipliers exhibit values from 1.96 (SE/RE = 0), 1.769 (SE/RE = 0.25), 1.68 (SE/RE = 0.5), 1.651 (SE/RE = 0.75) TO 1.645 (SE/RE = 1). The respective probability values beyond +/- 1.96 sigma were 2.5%/2.5%, 1.165%/3.849%, 0.368%/4.645%, 0.081%/4.934%, and 0.013%/5% for the SE/RE ratios of 0, 0.25, 0.5, 0.75, and 1. CONCLUSIONS: The results show that at SE/RE ratios > 0.75 the one-sided 95% probability level is practically reached. The results allow a refined calculation of TE at specified SE/RE ratios and a general understanding of the relevance of two- and one-sided probabilities at different SE/RE ratios.

Introduction of non-linearity by data transformation in method comparison and commutability studies.

BACKGROUND: Logarithmic transformation is recommended in method comparison or commutability studies when the standard deviation of the measurement results is heteroscedastic. We show that in the case of a considerable constant difference in the relationship between the x- and y-data, logarithmic transformation introduces non-linearity. METHODS: We used a simulated bivariate dataset [n=50; no systematic differences between the x- and y-data; x-data without error and y-data with concentration-dependent random, normally distributed error (CV=7%)], from which we generated two new sets of data: one by i) multiplying the y-data by 1.1, and the second by ii) adding a constant value of 15 to the y-data. RESULTS: The runs test (p<0.001) confirms that logarithmic transformation of the second dataset introduces non-linearity. Consequently, applying a linear regression model to the transformed data would result in erroneous decisions about commutability and in erroneously high estimates of the limits of agreement in method comparison studies. CONCLUSIONS: We recommend applying a linearity test after logarithmic transformation of bivariate data and, if necessary, to calculate the prediction intervals of a non-linear regression function.

Use of frozen sera for FT4 standardization: investigation by equilibrium dialysis combined with isotope dilution-mass spectrometry and immunoassay.

BACKGROUND: Serum-free thyroxine (FT4) testing is recommended for diagnosis or monitoring of thyroid dysfunction, particularly in cases of hormone binding abnormalities. However, the poor intermethod agreement among commercial FT4 assays suggests a need for standardization with a hierarchically higher measurement procedure. To that purpose, we applied equilibrium dialysis (ED) in combination with isotope dilution-liquid chromatography/tandem mass spectrometry (ID-LC-tandem MS). METHODS: After ED, we collected dialysate into tubes containing [13C6]-T4 for ID and [13C9]-T4 as carrier, purified the samples by solid-phase extraction, and analyzed them with LC/tandem MS. We evaluated the procedure's analytical performance and tested its suitability for measurement of hypo-, eu-, and hyperthyroid serum FT4 concentrations. We conducted a pilot method comparison study with 3 commercial assays to investigate whether frozen sera could be used for the purpose of FT4 standardization. RESULTS: The within-run, between-run, and total CVs (inclusive ED) were 3.7%, 4.2%, and 5.6%, respectively (17.7 pmol/L; n = 20). The mean accuracy, estimated from recovery experiments with dialysate and dialysis buffer supplemented at 8.7, 18.7, and 33.5 pmol/L, and from analysis of certified sera gravimetrically diluted to 9.8, 19.2, and 34.8 pmol/L, was 98.0% to 102.8%. The procedure's limit of detection and limit of quantification were 0.5 and 1.3 pmol/L, respectively. The method comparison demonstrated the suitability of the selected sera for standardization of FT4 assays and confirmed the lack of assay comparability. CONCLUSIONS: We demonstrated that the described ED-ID-LC/tandem MS procedure and the selected type of sera qualify for standardization of FT4 measurements.

Feasibility of standardization of serum C-peptide immunoassays with isotope-dilution liquid chromatography-tandem mass spectrometry.

BACKGROUND: Serum C-peptide concentrations reflect pancreatic function in different clinical and diagnostic settings; however, the utility of C-peptide testing is limited by the lack of standardized commercial immunoassays. Standardization can best be done by split-sample comparison with a hierarchically higher reference measurement procedure with a set of native sera. For serum peptides, isotope-dilution liquid chromatography-mass spectrometry (ID-LC/MS) is recommended as a reference measurement procedure. METHODS: We evaluated the analytical performance characteristics of an ID-LC/tandem MS procedure for measurement of serum C-peptide after a 2-step solid-phase extraction. To investigate the feasibility of this procedure for use in standardization, we also performed a method comparison with 3 representative commercial assays. RESULTS: The ID-LC/tandem MS procedure showed maximum within-run, between-run, and total CVs on dedicated sera (C-peptide concentrations, 1.6 and 4.0 mug/L) of 2.1%, 2.5%, and 2.9%, respectively; an accuracy of 94.6%-104.1%; a minimum trueness of 98.1% (95% confidence interval, 96.2%-100.0%), and limits of quantification and detection of 0.15 and 0.03 mug/L, respectively. Deming linear regression analysis of the method-comparison data showed that the immunoassays correlated well with ID-MS and were specific, but lacked intercomparability and trueness. We propose that the deficiencies can be resolved by recalibration on the basis of the method comparison. CONCLUSIONS: The ID-LC/tandem MS procedure is suitable for specific and accurate measurement of basal and stimulated serum concentrations of proinsulin C-peptide fragment 33-63 and is suitable for use in standardization of C-peptide immunoassays.