Simulation Models of Misclassification Error for Single Thresholds of High-Sensitivity Cardiac Troponin I Due to Assay Bias and Imprecision.

BACKGROUND: Clinical outcome studies for cardiac troponins (cTn) are expensive and difficult to design owing to variation in patients, in the assays, and in the incidence of different types of myocardial infarction (MI). To overcome these difficulties, simulation models were used to estimate the rate of misclassification error for MI and risk prediction resulting from assay bias and imprecision. METHODS: Finite mixture analysis of Abbott high-sensitivity cTnI (hs-cTnI) results at time 0 h in patients presenting early with acute coronary syndrome (ACS) symptoms to the emergency department (ED) [n = 145, Reducing the Time Interval for Identifying New Guideline (RING) study] allowed derivation of a simulation data set (n = 10000). hs-cTnI concentrations were modified by addition of bias or imprecision error. The percentage of all 10000 modified hs-cTnI results that were misclassified for MI at thresholds of 2, 5, 26.2, and 52 ng/L was determined by Monte Carlo simulation. Analyses were replicated with an all-comer emergency department (ED) population (n = 1137) ROMI (Optimum Troponin Cutoffs for ACS in the ED) study. RESULTS: In the RING study, simulation at 26.2-ng/L (99th percentile) and 52-ng/L thresholds were affected by both bias ±2 ng/L and imprecision (10%-20%) and had misclassification rates of 0.4% to 0.6%. Simulations at the 2-ng/L and 5-ng/L thresholds were only affected by bias. Misclassification rates at bias of ±1 ng/L were 10% for the 2-ng/L threshold, and 5% for the 5-ng/L threshold. CONCLUSIONS: Simulation models predicted that hs-cTnI results are seldom misclassified (<1% of patients) when interpretative thresholds are near or exceed the overall 99th percentile. However, simulation models also predicted that low hs-cTnI results, as recommended in guidelines, are prone to misclassification of 5%-10% of patients.

A Statistical Simulation to Evaluate the Robustness of Hb A1c Measurement in the Presence of Quantitative Error.

BACKGROUND: The performance requirements for hemoglobin (Hb) A1c analysis have been questioned as analytic methods have improved. We developed a statistical simulation that relates error to the clinical utility of an oft-used laboratory test, as a means of assessing test performance expectations. METHODS: Finite mixture modeling of the Centers for Disease Control and Prevention-National Health and Nutrition Examination Survey (NHANES) 2017-2020 Hb A1c data in conjunction with Monte Carlo sampling were used to model and simulate a population prior to the introduction of error into the results. The impact of error on clinical utility was assessed by categorizing the results using the American Diabetes Association (ADA) diagnostic criteria and assessing the sensitivity and specificity of Hb A1c under various degrees of error (bias and imprecision). RESULTS: With the current allowable total error threshold of 6% for Hb A1c measurement, the simulation estimated a worst case between 50% and 60% for both test sensitivity and specificity for the non-diabetic category. Similarly, sensitivity and specificity estimates for the pre-diabetic category were 30% to 40% and 60% to 70%, respectively. Finally, estimates for the diabetic category yielded values of 80% to 90% for sensitivity and >90% for specificity. CONCLUSIONS: Bias and imprecision greatly affect the clinical utility of Hb A1c for all patient groups. The simulated error demonstrated in this modeling impacts 3 critical applications of the Hb A1c in diabetes management: the capacity to reliably screen, diagnostic accuracy, and utility in diabetes monitoring.

Estimated Risk for Insulin Dose Error Among Hospital Patients Due to Glucose Meter Hematocrit Bias in 2020.

CONTEXT.­: Glycemic control requires accurate blood glucose testing. The extent of hematocrit interference is difficult to assess to assure quality patient care. OBJECTIVE.­: To predict the effect of patient hematocrit on the performance of a glucose meter and its corresponding impact on insulin-dosing error. DESIGN.­: Multilevel mixed regression was conducted to assess the extent that patient hematocrit influences Roche Accu-Chek Inform II glucose meters, using the Radiometer ABL 837 as a reference method collected during validation of 35 new meters. Regression coefficients of fixed effects for reference glucose, hematocrit, an interaction term, and random error were applied to 4 months of patient reference method results extracted from the laboratory information system. A hospital inpatient insulin dose algorithm was used to determine the frequency of insulin dose error between reference glucose and meter glucose results. RESULTS.­: Fixed effects regression for method and hematocrit predicted biases to glucose meter results that met the "95% within ±12%" for the US Food and Drug Administration goal, but combinations of fixed and random effects exceeded that target in emergency and hospital inpatient units. Insulin dose errors were predicted from the meter results. Twenty-eight percent of intensive care unit, 20.8% of hospital inpatient, and 17.7% of emergency department results were predicted to trigger a ±1 insulin dose error by fixed and random effects. CONCLUSIONS.­: The current extent of hematocrit interference on glucose meter performance is anticipated to cause insulin error by 1-dose category, which is likely associated with low patient risk.

An insulin-dose error assessment grid: A new tool to evaluate glucose meter performance.

OBJECTIVE: To develop a tool to assess the clinical accuracy of glucose meter performance using an insulin dosing protocol to assess the frequency and extent of error in insulin dose categories. METHODS: Retrospective comparison of 1815 glucose meter and central laboratory glucose results obtained from 1698 critically ill patients was conducted using the Parkes error grid, Surveillance error grid and an insulin dose error assessment grid with a sliding scale insulin dosing protocol used to manage critically ill patients. RESULTS: Parkes error grid and Surveillance error grid analyses indicated little risk conferred with the glucose meter results. Insulin dose error assessment grid complemented the aforementioned consensus error grids by determining quantifiable metrics, insulin dose category errors. Insulin dose error analysis indicated that 76.8% (1395/1815) would not have any change in insulin dose, 99.2% (1800/1815) within ±1 insulin dose category, 99.9% (1814/1815) within ±2 categories and 100% within ±3 insulin dose categories. CONCLUSIONS: Analysis with an insulin dose error grid provides information about the frequency and extent of insulin dose category errors with a specific insulin dosing protocol and describes potential clinical impact of glucose meter error.

Application of a Simulation Model to Estimate Treatment Error and Clinical Risk Derived from Point-of-Care International Normalized Ratio Device Analytic Performance.

BACKGROUND: In 2016, the Food and Drug Administration (FDA) proposed to enhance performance expectations for point-of-care testing (POCT) international normalized ratio (INR) devices relative to International Organization for Standardization (ISO) 17593:2007. The objective of the study was to estimate the frequency of warfarin dosing errors associated with a central laboratory INR method, a POCT INR method, and the proposed FDA performance goals. METHODS: A data set of INR results (n = 51912) from adult patients with INR ≤4 was used to assess the influence of adding assay imprecision and bias on warfarin dose decisions. The frequency of error in warfarin dose and size of error (≥1 or ≥2 dose categories) was compared using published assay specifications for the Instrumentation Laboratory ACL TOP® and the Roche Diagnostics CoaguChek® XS relative to the proposed FDA guidelines. RESULTS: The frequency of warfarin dose misclassification was largely influenced by bias and was not sensitive to assay imprecision. The central laboratory and POCT INR methods met the FDA performance specifications, had equal rates of ≥2 warfarin dose category error, and had statistically different rates of ≥1 warfarin dose category error in large samples (n >250). CONCLUSIONS: Simulation models are useful tools for evaluating POCT INR assay performance criteria required to achieve the proposed FDA guidelines. This simulation depicted how the Roche Diagnostics CoaguChek XS instrument meets the guideline.