Large-Scale Scientific Study Led by a Professional Organization during the COVID-19 Pandemic: Operations, Best Practices, and Lessons Learned.

In 2021, the Association for Diagnostics & Laboratory Medicine (ADLM) (formerly the American Association for Clinical Chemistry [AACC]) developed a scientific study that aimed to contribute to the understanding of SARS-CoV-2 immunity during the evolving course of the pandemic. This study was led by a group of expert member volunteers and resulted in survey data from 975 individuals and blood collection from 698 of those participants. This paper describes the formulation and execution of this large-scale scientific study, encompassing best practices and insights gained throughout the endeavor.

An updated protocol based on CLSI document C37 for preparation of off-the-clot serum from individual units for use alone or to prepare commutable pooled serum reference materials.

Manufacturers of in vitro diagnostic medical devices, clinical laboratories, research laboratories and calibration laboratories require commutable reference materials that can be used in the calibration hierarchies of medical laboratory measurement procedures used for human specimens to establish metrological traceability to higher order reference systems. Commutable materials are also useful in external quality assessment surveys. In order to achieve these goals, matrix-based reference materials with long-term stability, appropriate measurand concentrations and commutability with individual human specimens are required. The Clinical and Laboratory Standards Institute (CLSI) guideline C37-A (now archived) provided guidance to prepare commutable pooled serum reference materials for use in the calibration hierarchies of cholesterol measurement procedures. Experience using the C37-A guideline has identified a number of technical enhancements as well as applications to measurands other than cholesterol. This experience is incorporated into this updated protocol to ensure the procedure will continue to meet the needs of the medical laboratory. The updated protocol describes a procedure for preparing frozen human serum units or pools with minimal matrix alterations that are likely to be commutable with individual human serum samples. The protocol provides step-by-step guidance for the planning phase, collection of individual serum units, processing the units, qualifying the units for use in a pool and frozen storage of aliquots of pooled sera to manufacture frozen serum pools. Guidance on how to perform quality control of the final product and suggestions on documentation are also provided.

Baseline Assessment of 25-Hydroxyvitamin D Reference Material and Proficiency Testing/External Quality Assurance Material Commutability: A Vitamin D Standardization Program Study.

The Vitamin D Standardization Program (VDSP) coordinated a study in 2012 to assess the commutability of reference materials and proficiency testing/external quality assurance materials for total 25-hydroxyvitamin D [25(OH)D] in human serum, the primary indicator of vitamin D status. A set of 50 single-donor serum samples as well as 17 reference and proficiency testing/external quality assessment materials were analyzed by participating laboratories that used either immunoassay or LC-MS methods for total 25(OH)D. The commutability test materials included National Institute of Standards and Technology Standard Reference Material 972a Vitamin D Metabolites in Human Serum as well as materials from the College of American Pathologists and the Vitamin D External Quality Assessment Scheme. Study protocols and data analysis procedures were in accordance with Clinical and Laboratory Standards Institute guidelines. The majority of the test materials were found to be commutable with the methods used in this commutability study. These results provide guidance for laboratories needing to choose appropriate reference materials and select proficiency or external quality assessment programs and will serve as a foundation for additional VDSP studies.

Baseline Assessment of 25-Hydroxyvitamin D Assay Performance: A Vitamin D Standardization Program (VDSP) Interlaboratory Comparison Study.

The Vitamin D Standardization Program (VDSP) coordinated an interlaboratory study to assess the comparability of measurements of total 25-hydroxyvitamin D [25(OH)D] in human serum, which is the primary marker of vitamin D status. A set of 50 individual donor samples were analyzed by 15 different laboratories representing national nutrition surveys, assay manufacturers, and clinical and/or research laboratories to provide results for total 25(OH)D using both immunoassays (IAs) and LC tandem MS (MS/MS). The results were evaluated relative to bias compared with the target values assigned based on a combination of measurements at Ghent University (Belgium) and the U.S. National Institute of Standards and Technology using reference measurement procedures for the determination of 25(OH)D2 and 25(OH)D3. CV and mean bias for each laboratory and assay platform were assessed and compared with previously established VDSP performance criteria, namely CV ≤ 10% and mean bias ≤ 5%. Nearly all LC-MS/MS results achieved VDSP criteria, whereas only 50% of IAs met the criterion for a ≤10% CV and only three of eight IAs achieved the ≤5% bias. These results establish a benchmark for the evaluation of 25(OH)D assay performance and standardization activities in the future.

The Vitamin D Standardization Program (VDSP) Manual for Retrospective Laboratory Standardization of Serum 25-Hydroxyvitamin D Data.

Low concentrations of total 25-hydroxyvitamin D [25(OH)D], the principal biological measure of vitamin D status, have been associated with clinical and public health outcomes. The determination of levels under which there is an increase in the risk of disease, as well as comparisons across populations, have been difficult to establish due the large assay variability in measuring 25(OH)D. Accordingly, the Vitamin D Standardization Program (VDSP) includes the retrospective standardization of existing 25(OH)D values collected by epidemiological and clinical studies, as well as clinical trials, as one of its main objectives. We introduce methodology developed by the VDSP that can be used to standardize the measurement of time-stable analytes, including 25(OH)D, in samples that have been banked and maintained appropriately. Sample size estimation formulae are first applied to calculate the required number of banked blood samples to be reanalyzed using either of two approaches. In the first approach, existing samples are remeasured using the current measurement procedure, and an equation relating "old" to "current" measurements is obtained. A second set of sera, usually 40-50 single-donor serum samples, are measured with the current measurement procedure and an assay traceable to a reference measurement procedure and/or certified reference materials, which yields a second calibration equation. These two equations are combined to produce standardized levels from the original old values. This approach is necessary when study restrictions prevent serum samples from being shipped to an external laboratory and is illustrated with samples from the Canadian Health Measures Survey. When serum samples are permitted to be shared with other laboratories, or the study investigators can carry out the measurements with a traceable assay, a single calibration equation method is used. Existing samples are selected and remeasured using the available traceable assay. We outline the statistical theory supporting the VDSP protocol and provide implementation examples. The methods proposed are generalizable to any instance in which banked specimens have been properly prepared and stored and the analyte of interest is stable under those conditions.

Traceability in laboratory medicine: a global driver for accurate results for patient care.

Laboratory medicine results influence a high percentage of all clinical decisions. Globalization requires that laboratory medicine results should be transferable between methods in the interests of patient safety. International collaboration is necessary to deliver this requirement. That collaboration should be based on traceability in laboratory medicine and the adoption of higher order international commutable reference materials and measurement procedures. Application of the metrological traceability chain facilitates a universal approach. The measurement of serum cholesterol and blood HbA1c serve as examples of the process of method standardization where an impact on clinical outcomes is demonstrable. The measurement of plasma parathyroid hormone and blood HbA2 serve as examples where the current between-method variability is compromising patient management and method standardization and/or harmonization is required. Challenges to the widespread adoption of traceability in laboratory medicine include the availability of reference materials and methods, geographical differences, the use of variable units, complex analytes and limited global coordination. The global collaboration requires the involvement of several different stakeholder groups ranging from international experts to laboratory medicine specialists in routine clinical laboratories. A coordinated action plan is presented with actions attributable to each of these stakeholder groups.

Creation of a Universal Sample Bank for Determining the 99th Percentile for Cardiac Troponin Assays.

BACKGROUND: International guidelines authored and endorsed by professional societies of cardiology, emergency medicine, and laboratory medicine are unanimous that the cutoff concentration for establishing a diagnosis of acute myocardial infarction be set at the 99th percentile of a healthy population. The establishment of the actual 99th percentile value is assay- and sample-dependent and is influenced by the population of individuals selected for testing. We created a sample bank that will enable manufacturers of troponin assays a consistent comparison of the 99th percentile. METHODS: Participants were recruited from those attending the 2015 Annual Meeting of the AACC for the creation of a universal sample bank of apparently healthy individuals (free from uncontrolled diabetes, renal insufficiency, and heart disease). For those who met eligibility criteria and signed a written consent to participate, 60 mL blood was collected into 6 10-mL tubes each (2 serum, 3 heparin plasma, and 1 EDTA). Whole blood was tested for hemoglobin (Hb) A1c, and serum was tested for N-terminal pro-B-type natriuretic peptide (NT-proBNP) and creatinine. RESULTS: There were a total of 764 individuals who consented during the AACC Annual Meeting. After this initial enrollment, it was determined that there was an insufficient number of male participants recruited. Under the same protocol and consent, blood from 131 additional males was collected at the University of Maryland. Samples were centrifuged, and 240 µL aliquots of the 2 serum, 3 heparin plasma, and 1 EDTA tubes were frozen at 70 °C within 2 h of collection. The labeled samples were divided into boxes containing 1 aliquot from each individual. Sets of these samples were made available for purchase to manufacturers of cardiac troponin assays. Eighty-eight samples were excluded from the database for having a high NT-proBNP (>300 ng/L), low estimated glomerular filtration rate (eGFR) (<60 mL/min/1.73 m2), high Hb A1c (≥6.5%), or preanalytical sample issues and consenting/data issues. The final total was 808 individuals (402 females and 406 males; 60% Caucasian, 26% African or African American, 11% Asian or Pacific Islander, and 3% other). CONCLUSIONS: The creation of a bank of samples from healthy individuals enables a consistent comparison of the 99th percentile results from manufacturers of cardiac troponin assays.

The International Consortium for Harmonization of Clinical Laboratory Results (ICHCLR) - A Pathway for Harmonization.

Results from clinical laboratory measurement procedures must be equivalent to enable effective use of clinical guidelines for disease diagnosis and patient management. Analytical results that are harmonized and independent of the measurement system, time, and location of testing is essential for providing adequate patient care. The key to generating harmonized results is establishing traceability to an accepted reference standard where available. Awareness of the benefits of having traceable measurement results that are harmonized has increased along with efforts to develop approaches to enable and facilitate the implementation of harmonization. Although several organizations are addressing harmonization of test procedures, centralized and cooperative global oversight is needed to ensure that the most important tests are being addressed and resources are optimally used. Working with its domestic and international partners, the American Association for Clinical Chemistry (AACC) has created an International Consortium for Harmonization of Clinical Laboratory Results. Advances in this area will improve the quality of patient care.

Current practices and challenges in the standardization and harmonization of clinical laboratory tests.

Effective patient care, clinical research, and public health efforts require comparability of laboratory results independent of time, place, and measurement procedure. Comparability is achieved by establishing metrological traceability, which ensures that measurement procedures measure the same quantity and that the calibration of measurement procedures is traceable to a common reference system consisting of reference methods and materials. Whereas standardization ensures traceability to the International System of Units, harmonization ensures traceability to a reference system agreed on by convention. This article provides an overview of standardization and harmonization with an emphasis on commutability as an important variable that affects testing accuracy. Commutability of reference materials is required to ensure that traceability is established appropriately and that laboratory results are comparable. The use of noncommutable reference materials leads to inaccurate results. Whereas procedures and protocols for standardizing measurements are established and have been successfully applied in efforts such as the Hormones Standardization Program of the CDC, harmonization activities require new, more complex procedures and approaches. The American Association for Clinical Chemistry, together with its domestic and international partners, formed the International Consortium for Harmonization of Clinical Laboratory Results to coordinate harmonization efforts. Reference systems, as well as procedures and protocols to establish traceability of clinical laboratory tests, have been established and continue to be developed by national and international groups and organizations. Serum tests of thyroid function, including those for the thyroid hormones thyroxine and triiodothyronine, are among the clinical procedures for which standardization efforts are well under way. Approaches to the harmonization of measurement procedures for serum concentrations of thyroid-stimulating hormone are likewise under development.

Difference in bias approach for commutability assessment: application to frozen pools of human serum measured by 8 direct methods for HDL and LDL cholesterol.

BACKGROUND: We used a difference in bias approach to evaluate the commutability of 4 frozen serum pools for 8 direct methods for measurement of HDL and LDL cholesterol (HDLC and LDLC). METHODS: Freshly collected nonfrozen sera from 138 diseased and 37 nondiseased patients and 4 frozen pools from the CDC Lipid Standardization Program were measured by direct methods and by the beta-quantification reference measurement procedure of the CDC. We used an error components model to estimate the difference in the bias component of error plus its uncertainty for frozen pools vs patient samples between the direct method and the reference procedure. Frozen pools with bias differences less than a critical value determined by either medical requirements for bias or the random error components of the measurement procedures were considered commutable. RESULTS: On the basis of medical requirement criteria, 1 of the 4 frozen pools was commutable for most of the HDLC methods for both diseased and nondiseased patients, and none was commutable for LDLC methods. On the basis of random error criteria, all of the frozen pools were generally commutable for all of the HDLC methods for both diseased and nondiseased patients, and 1 of the 4 frozen pools was generally commutable for most of the LDLC methods for both diseased and nondiseased patients. CONCLUSIONS: Commutability was assessed as the closeness of agreement of the difference in bias between a reference material and a set of patient samples. Criteria for commutability could be based on fixed medical requirements for bias or on random error components.

Evaluation of four different equations for calculating LDL-C with eight different direct HDL-C assays.

BACKGROUND: Low-density lipoprotein cholesterol (LDL-C) is often calculated (cLDL-C) by the Friedewald equation, which requires high-density lipoprotein cholesterol (HDL-C) and triglycerides (TG). Because there have been considerable changes in the measurement of HDL-C with the introduction of direct assays, several alternative equations have recently been proposed. METHODS: We compared 4 equations (Friedewald, Vujovic, Chen, and Anandaraja) for cLDL-C, using 8 different direct HDL-C (dHDL-C) methods. LDL-C values were calculated by the 4 equations and determined by the ß quantification reference method procedure in 164 subjects. RESULTS: For normotriglyceridemic samples (TG<200mg/dl), between 6.2% and 24.8% of all results exceeded the total error goal of 12% for LDL-C, depending on the dHDL-C assay and cLDL-C equation used. Friedewald equation was found to be the optimum equation for most but not all dHDL-C assays, typically leading to less than 10% misclassification of cardiovascular risk based on LDL-C. Hypertriglyceridemic samples (>200mg/dl) showed a large cardiovascular risk misclassification rate (30%-50%) for all combinations of dHDL-C assays and cLDL-C equations. CONCLUSION: The Friedewald equation showed the best performance for estimating LDL-C, but its accuracy varied considerably depending on the specific dHDL-C assay used. None of the cLDL-C equations performed adequately for hypertriglyceridemic samples.

Roadmap for harmonization of clinical laboratory measurement procedures.

Results between different clinical laboratory measurement procedures (CLMP) should be equivalent, within clinically meaningful limits, to enable optimal use of clinical guidelines for disease diagnosis and patient management. When laboratory test results are neither standardized nor harmonized, a different numeric result may be obtained for the same clinical sample. Unfortunately, some guidelines are based on test results from a specific laboratory measurement procedure without consideration of the possibility or likelihood of differences between various procedures. When this happens, aggregation of data from different clinical research investigations and development of appropriate clinical practice guidelines will be flawed. A lack of recognition that results are neither standardized nor harmonized may lead to erroneous clinical, financial, regulatory, or technical decisions. Standardization of CLMPs has been accomplished for several measurands for which primary (pure substance) reference materials exist and/or reference measurement procedures (RMPs) have been developed. However, the harmonization of clinical laboratory procedures for measurands that do not have RMPs has been problematic owing to inadequate definition of the measurand, inadequate analytical specificity for the measurand, inadequate attention to the commutability of reference materials, and lack of a systematic approach for harmonization. To address these problems, an infrastructure must be developed to enable a systematic approach for identification and prioritization of measurands to be harmonized on the basis of clinical importance and technical feasibility, and for management of the technical implementation of a harmonization process for a specific measurand.

Proposed serum cholesterol reference measurement procedure by gas chromatography-isotope dilution mass spectrometry.

BACKGROUND: Our purpose was to establish a mass spectrometry reference measurement procedure (RMP) for cholesterol to use in the CDC's standardization programs. We explored a gas chromatography-isotope dilution mass spectrometry (GC-IDMS) procedure using a multilevel standard calibration curve to quantify samples with varying cholesterol concentrations. METHODS: We calibrated the mass spectrometry instrument by isotope dilution with a pure primary standard reference material and an isotopically enriched cholesterol analog as the internal standard (IS). We diluted the serum samples with Tris-HCl buffer (pH 7.4, 0.05 mol/L, 0.25% Triton X-100) before analysis. We used 17 serum pools, 10 native samples, and 2 standard reference materials (SRMs). We compared the GC-IDMS measurements with the CDC's modified Abell-Levy-Brodie-Kendall (AK) RMP measurements and assessed method accuracy by analyzing 2 SRMs. We evaluated the procedure for lack of interference by analyzing serum spiked with a mixture of 7 sterols. RESULTS: The mean percent bias between the AK and the GC-IDMS RMP was 1.6% for all samples examined. The mean percent bias from NIST's RMP was 0.5% for the SRMs. The total %CVs for SRM 1951b levels I and II were 0.61 and 0.73%, respectively. We found that none of the sterols investigated interfered with the cholesterol measurement. CONCLUSIONS: The low imprecision, linear response, lack of interferences, and acceptable bias vs the NIST primary RMP qualifies this procedure as an RMP for determining serum cholesterol. The CDC will adopt and implement this GC-IDMS procedure for cholesterol standardization.

Non-HDL cholesterol shows improved accuracy for cardiovascular risk score classification compared to direct or calculated LDL cholesterol in a dyslipidemic population.

BACKGROUND: Our objective was to evaluate the accuracy of cardiovascular disease (CVD) risk score classification by direct LDL cholesterol (dLDL-C), calculated LDL cholesterol (cLDL-C), and non-HDL cholesterol (non-HDL-C) compared to classification by reference measurement procedures (RMPs) performed at the CDC. METHODS: We examined 175 individuals, including 138 with CVD or conditions that may affect LDL-C measurement. dLDL-C measurements were performed using Denka, Kyowa, Sekisui, Serotec, Sysmex, UMA, and Wako reagents. cLDL-C was calculated by the Friedewald equation, using each manufacturer's direct HDL-C assay measurements, and total cholesterol and triglyceride measurements by Roche and Siemens (Advia) assays, respectively. RESULTS: For participants with triglycerides<2.26 mmol/L (<200 mg/dL), the overall misclassification rate for the CVD risk score ranged from 5% to 17% for cLDL-C methods and 8% to 26% for dLDL-C methods when compared to the RMP. Only Wako dLDL-C had fewer misclassifications than its corresponding cLDL-C method (8% vs 17%; P<0.05). Non-HDL-C assays misclassified fewer patients than dLDL-C for 4 of 8 methods (P<0.05). For participants with triglycerides≥2.26 mmol/L (≥200 mg/dL) and<4.52 mmol/L (<400 mg/dL), dLDL-C methods, in general, performed better than cLDL-C methods, and non-HDL-C methods showed better correspondence to the RMP for CVD risk score than either dLDL-C or cLDL-C methods. CONCLUSIONS: Except for hypertriglyceridemic individuals, 7 of 8 dLDL-C methods failed to show improved CVD risk score classification over the corresponding cLDL-C methods. Non-HDL-C showed overall the best concordance with the RMP for CVD risk score classification of both normal and hypertriglyceridemic individuals.

Validation of CKD and related conditions in existing data sets: A systematic review.

BACKGROUND: Accurate classification of individuals with kidney disease is vital to research and public health efforts aimed at improving health outcomes. Our objective is to identify and synthesize published literature evaluating the accuracy of existing data sources related to kidney disease. STUDY DESIGN: A systematic review of studies seeking to validate the accuracy of the underlying data relevant to kidney disease. SETTING & POPULATION: US-based and international studies covering a wide range of both outpatient and inpatient study populations. SELECTION CRITERIA FOR STUDIES: Any English-language study investigating the prevalence or cause of kidney disease, existence of comorbid conditions, or cause of death in patients with chronic kidney disease (CKD). All definitions and stages of CKD, including end-stage renal disease (ESRD), were accepted. INDEX TESTS: Presence of a kidney disease-related variable in existing data sets, including administrative data sets and disease registries. REFERENCE TESTS: Presence of a kidney disease-related variable defined using laboratory criteria or medical record review. RESULTS: 30 studies were identified. Most studies investigated the accuracy of kidney disease reporting, comparing coded renal disease with that defined using estimated glomerular filtration rate. The sensitivity of coded renal disease varied widely (0.08-0.83). Specificity was higher, with all studies reporting values ≥0.90. Studies evaluating the cause of CKD, comorbid conditions, and cause of death in patients with CKD used ESRD or transplant populations exclusively, and accuracy was highly variable compared with ESRD registry data. LIMITATIONS: Only English-language studies were evaluated. CONCLUSIONS: Given the heterogeneous results of validation studies, a variety of attributes of existing data sources, including the accuracy of individual data items within these sources, should be considered carefully before use in research, quality improvement, and public health efforts.

Seven direct methods for measuring HDL and LDL cholesterol compared with ultracentrifugation reference measurement procedures.

BACKGROUND: Methods from 7 manufacturers and 1 distributor for directly measuring HDL cholesterol (C) and LDL-C were evaluated for imprecision, trueness, total error, and specificity in nonfrozen serum samples. METHODS: We performed each direct method according to the manufacturer's instructions, using a Roche/Hitachi 917 analyzer, and compared the results with those obtained with reference measurement procedures for HDL-C and LDL-C. Imprecision was estimated for 35 runs performed with frozen pooled serum specimens and triplicate measurements on each individual sample. Sera from 37 individuals without disease and 138 with disease (primarily dyslipidemic and cardiovascular) were measured by each method. Trueness and total error were evaluated from the difference between the direct methods and reference measurement procedures. Specificity was evaluated from the dispersion in differences observed. RESULTS: Imprecision data based on 4 frozen serum pools showed total CVs <3.7% for HDL-C and <4.4% for LDL-C. Bias for the nondiseased group ranged from -5.4% to 4.8% for HDL-C and from -6.8% to 1.1% for LDL-C, and for the diseased group from -8.6% to 8.8% for HDL-C and from -11.8% to 4.1% for LDL-C. Total error for the nondiseased group ranged from -13.4% to 13.6% for HDL-C and from -13.3% to 13.5% for LDL-C, and for the diseased group from -19.8% to 36.3% for HDL-C and from -26.6% to 31.9% for LDL-C. CONCLUSIONS: Six of 8 HDL-C and 5 of 8 LDL-C direct methods met the National Cholesterol Education Program total error goals for nondiseased individuals. All the methods failed to meet these goals for diseased individuals, however, because of lack of specificity toward abnormal lipoproteins.