Validating, Implementing, and Monitoring Machine Learning Solutions in the Clinical Laboratory Safely and Effectively.

BACKGROUND: Machine learning solutions offer tremendous promise for improving clinical and laboratory operations in pathology. Proof-of-concept descriptions of these approaches have become commonplace in laboratory medicine literature, but only a scant few of these have been implemented within clinical laboratories, owing to the often substantial barriers in validating, implementing, and monitoring these applications in practice. This mini-review aims to highlight the key considerations in each of these steps. CONTENT: Effective and responsible applications of machine learning in clinical laboratories require robust validation prior to implementation. A comprehensive validation study involves a critical evaluation of study design, data engineering and interoperability, target label definition, metric selection, generalizability and applicability assessment, algorithmic fairness, and explainability. While the main text highlights these concepts in broad strokes, a supplementary code walk-through is also provided to facilitate a more practical understanding of these topics using a real-world classification task example, the detection of saline-contaminated chemistry panels.Following validation, the laboratorian's role is far from over. Implementing machine learning solutions requires an interdisciplinary effort across several roles in an organization. We highlight the key roles, responsibilities, and terminologies for successfully deploying a validated solution into a live production environment. Finally, the implemented solution must be routinely monitored for signs of performance degradation and updated if necessary. SUMMARY: This mini-review aims to bridge the gap between theory and practice by highlighting key concepts in validation, implementation, and monitoring machine learning solutions effectively and responsibly in the clinical laboratory.

Limited Evidence for Use of a Black Race Modifier in eGFR Calculations: A Systematic Review.

BACKGROUND: Commonly used estimated glomerular filtration rate (eGFR) equations include a Black race modifier (BRM) that was incorporated during equation derivation. Race is a social construct, and a poorly characterized variable that is applied inconsistently in clinical settings. The BRM results in higher eGFR for any creatinine concentration, implying fundamental differences in creatinine production or excretion in Black individuals compared to other populations. Equations without inclusion of the BRM have the potential to detect kidney disease earlier in patients at the greatest risk of chronic kidney disease (CKD), but also has the potential to over-diagnose CKD or impact downstream clinical interventions. The purpose of this study was to use an evidence-based approach to systematically evaluate the literature relevant to the performance of the eGFR equations with and without the BRM and to examine the clinical impact of the use or removal. CONTENT: PubMed and Embase databases were searched for studies comparing measured GFR to eGFR in racially diverse adult populations using the Modification of Diet in Renal Disease or the 2009-Chronic Kidney Disease Epidemiology Collaboration-creatinine equations based on standardized creatinine measurements. Additionally, we searched for studies comparing clinical use of eGFR calculated with and without the BRM. Here, 8632 unique publications were identified; an additional 3 studies were added post hoc. In total, 96 studies were subjected to further analysis and 44 studies were used to make a final assessment. SUMMARY: There is limited published evidence to support the use of a BRM in eGFR equations.

An international multi-center serum protein electrophoresis accuracy and M-protein isotyping study. Part II: limit of detection and follow-up of patients with small M-proteins.

Background Electrophoretic methods to detect, characterize and quantify M-proteins play an important role in the management of patients with monoclonal gammopathies (MGs). Significant uncertainty in the quantification and limit of detection (LOD) is documented when M-proteins are <10 g/L. Using spiked sera, we aimed to assess the variability in intact M-protein quantification and LOD across 16 laboratories. Methods Sera with normal, hypo- or hyper-gammaglobulinemia were spiked with daratumumab or elotuzumab, with concentrations from 0.125 to 10 g/L (n = 62) along with a beta-migrating sample (n = 9). Laboratories blindly analyzed samples according to their serum protein electrophoresis (SPEP)/isotyping standard operating procedures. LOD and intra-laboratory percent coefficient of variation (%CV) were calculated and further specified with regard to the method (gel/capillary electrophoresis [CZE]), gating strategy (perpendicular drop [PD]/tangent skimming [TS]), isotyping (immunofixation/immunosubtraction [ISUB]) and manufacturer (Helena/Sebia). Results All M-proteins ≥1 g/L were detected by SPEP. With isotyping the LOD was moderately more sensitive than with SPEP. The intensity of polyclonal background had the biggest negative impact on LOD. Independent of the method used, the intra-laboratory imprecision of M-protein quantification was small (mean CV = 5.0%). Low M-protein concentration and high polyclonal background had the strongest negative impact on intra-laboratory precision. All laboratories were able to follow trend of M-protein concentrations down to 1 g/L. Conclusions In this study, we describe a large variation in the reported LOD for both SPEP and isotyping; overall LOD is most affected by the polyclonal immunoglobulin background. Satisfactory intra-laboratory precision was demonstrated. This indicates that the quantification of small M-proteins to monitor patients over time is appropriate, when subsequent testing is performed within the same laboratory.

An international multi-center serum protein electrophoresis accuracy and M-protein isotyping study. Part I: factors impacting limit of quantitation of serum protein electrophoresis.

Background Serum protein electrophoresis (SPEP) is used to quantify the serum monoclonal component or M-protein, for diagnosis and monitoring of monoclonal gammopathies. Significant imprecision and inaccuracy pose challenges in reporting small M-proteins. Using therapeutic monoclonal antibody-spiked sera and a pooled beta-migrating M-protein, we aimed to assess SPEP limitations and variability across 16 laboratories in three continents. Methods Sera with normal, hypo- or hypergammaglobulinemia were spiked with daratumumab, Dara (cathodal migrating), or elotuzumab, Elo (central-gamma migrating), with concentrations from 0.125 to 10 g/L (n = 62) along with a beta-migrating sample (n = 9). Provided with total protein (reverse biuret, Siemens), laboratories blindly analyzed samples according to their SPEP and immunofixation (IFE) or immunosubtraction (ISUB) standard operating procedures. Sixteen laboratories reported the perpendicular drop (PD) method of gating the M-protein, while 10 used tangent skimming (TS). A mean percent recovery range of 80%-120% was set as acceptable. The inter-laboratory %CV was calculated. Results Gamma globulin background, migration pattern and concentration all affect the precision and accuracy of quantifying M-proteins by SPEP. As the background increases, imprecision increases and accuracy decreases leading to overestimation of M-protein quantitation especially evident in hypergamma samples, and more prominent with PD. Cathodal migrating M-proteins were associated with less imprecision and higher accuracy compared to central-gamma migrating M-proteins, which is attributed to the increased gamma background contribution in M-proteins migrating in the middle of the gamma fraction. There is greater imprecision and loss of accuracy at lower M-protein concentrations. Conclusions This study suggests that quantifying exceedingly low concentrations of M-proteins, although possible, may not yield adequate accuracy and precision between laboratories.

A Survey of Cerebrospinal Fluid Total Protein Upper Limits in Canada: Time for an Update?

BACKGROUND: The antiquated standard reference range of 0.15-0.45 g/L for cerebrospinal fluid total protein (CSF-TP) is well entrenched in medical literature and laboratory operating procedures across the world. METHODS: We conducted a web-based survey with a response rate of 34.9% through the listserv of the Canadian Neurological Sciences Federation. Additional laboratory reference data were collated by telephone interview of hospital laboratory technologists across Canada. RESULTS: A total of 142 site responses were obtained: 64.1% from academic/tertiary hospitals and 35.9% from community hospitals. A strong majority (80.4%) of both types of institutions reported using a CSF-TP upper reference limit of 0.45 g/L or less. As a rule, no age adjustments were implemented in CSF-TP-level interpretation. CONCLUSIONS: Recent well-powered laboratory reference studies have documented CSF-TP upper reference limits that are above 0.6 g/L starting at age 50, with incremental limits partitioned by subsequent decades of age. The conventional 0.45 g/L limit could lead to false positive results. Our survey suggests there is a need to consider a wide adoption of data-driven, rather than historical, reference values.

Cerebrospinal Fluid Total Protein Reference Intervals Derived from 20 Years of Patient Data.

BACKGROUND: Reference intervals are vital for interpretation of laboratory results. Many existing reference intervals for cerebrospinal fluid total protein (CSF-TP) are derived from old literature because of the invasive nature of sampling. The objective of this study was to determine reference intervals for CSF-TP using available patient data. METHODS: Twenty years of hospital database information was mined for previously reported CSF-TP results. Associated demographic, laboratory, and clinical diagnosis (International Classification of Diseases 9/10 codes) details were extracted. CSF-TP results included 3 different analytical platforms: the Siemens Vista 1500, Beckman Lx20, and Roche Hitachi 917. From an initial data set of 19591 samples, the following exclusion criteria were applied: incomplete data, white blood cells (WBCs) >5 × 106/L, red blood cells (RBCs) >50 × 106/L, and glucose <2.5 mmol/L. Patient charts were reviewed in detail to exclude 60 different conditions for which increases in CSF-TP would be expected. A total of 6068 samples were included; 63% of the samples were from females. Continuous reference intervals were determined using quantile regression. Age- and sex-partitioned intervals were established using the quantile regression equation and splitting age-groups into 5-year bins. RESULTS: CSF-TP showed a marked age dependence, and males had significantly higher CSF-TP than females across all ages. CSF-TP results from the 3 different instruments and manufacturers showed small (approximately 0.04 g/L), but statistically significant, differences. CSF-TP showed weak, but again statistically significant, correlation with WBC and RBC but was independent of serum total protein and creatinine. CONCLUSIONS: The age dependence of CSF-TP supports that age-partitioned reference intervals will be more accurate than a single cutoff, particularly in patients with advancing age.

Selection of Single-Analyte Delta Check Rules with Logistic Regression for Detection of Intravenous Fluid Contamination in a Clinical Chemistry Laboratory.

BACKGROUND: The conventional single-analyte delta check, utilized for identifying intravenous fluid contamination and other preanalytical errors, is known to flag many specimens reflecting true patient status changes. This study aimed to derive delta check rules that more accurately identify contamination. METHODS: Results for calcium, creatinine, glucose, sodium, and potassium were retrieved from 326 103 basic or comprehensive metabolic panels tested between February 2021 and January 2022. In total, 7934 specimens showed substantial result changes, of which 1489 were labeled as either contaminated or non-contaminated based on chart review. These labeled specimens were used to derive logistic regression models and to select the most predictive single-analyte delta checks for 4 common contaminants. Their collective performance was evaluated using a test data set from October 2023 comprising 14 717 specimens. RESULTS: The most predictive single-analyte delta checks included a calcium change by ≤-24% for both saline and Plasma-Lyte A contamination, a potassium increase by ≥3.0 mmol/L for potassium contamination, and a glucose increase by ≥400 mg/dL (22.2 mmol/L) for dextrose contamination. In the training data sets, multi-analyte logistic regression models performed better than single-analyte delta checks. In the test data set, logistic regression models and single-analyte delta checks demonstrated collective alert rates of 0.58% (95% CI, 0.46%-0.71%) and 0.60% (95% CI, 0.49%-0.74%), respectively, along with collective positive predictive values of 79% (95% CI, 70%-89%) and 77% (95% CI, 68%-87%). CONCLUSIONS: Single-analyte delta checks selected by logistic regression demonstrated a low false alert rate.

Protein modification by deamidation indicates variations in joint extracellular matrix turnover.

As extracellular proteins age, they undergo and accumulate nonenzymatic post-translational modifications that cannot be repaired. We hypothesized that these could be used to systemically monitor loss of extracellular matrix due to chronic arthritic diseases such as osteoarthritis (OA). To test this, we predicted sites of deamidation in cartilage oligomeric matrix protein (COMP) and confirmed, by mass spectroscopy, the presence of deamidated (Asp(64)) and native (Asn(64)) COMP epitopes (mean 0.95% deamidated COMP (D-COMP) relative to native COMP) in cartilage. An Asp(64), D-COMP-specific ELISA was developed using a newly created monoclonal antibody 6-1A12. In a joint replacement study, serum D-COMP (p = 0.017), but not total COMP (p = 0.5), declined significantly after replacement demonstrating a joint tissue source for D-COMP. In analyses of 450 participants from the Johnston County Osteoarthritis Project controlled for age, gender, and race, D-COMP was associated with radiographic hip (p < 0.0001) but not knee (p = 0.95) OA severity. In contrast, total COMP was associated with radiographic knee (p < 0.0001) but not hip (p = 0.47) OA severity. D-COMP was higher in soluble proteins extracted from hip cartilage proximal to OA lesions compared with remote from lesions (p = 0.007) or lesional and remote OA knee (p < 0.01) cartilage. Total COMP in cartilage did not vary by joint site or proximity to the lesion. This study demonstrates the presence of D-COMP in articular cartilage and the systemic circulation, and to our knowledge, it is the first biomarker to show specificity for a particular joint site. We believe that enrichment of deamidated epitope in hip OA cartilage indicates a lesser repair response of hip OA compared with knee OA cartilage.

Evaluation of high resolution gel ß(2)-transferrin for detection of cerebrospinal fluid leak.

BACKGROUND: Cerebrospinal fluid (CSF) leaks are potentially life-threatening conditions that can be diagnosed by detection of ß(2)-transferrin using protein electrophoresis. Another less commonly available test is ß-trace protein quantitation using immunoassay. The objectives of this study were to evaluate a new immunofixation-based ß(2)-transferrin test for detection of CSF leaks and to compare it to an existing agarose gel electrophoresis test and ß-trace protein immunoassay. METHODS: For method comparison, 63 consecutive samples from physician-ordered ß(2)-transferrin tests were analyzed using two different electrophoresis methods, agarose gel fractionation followed by acid-violet staining, and high resolution agarose gel electrophoresis followed by ß(2)-transferrin immunofixation. A subset of samples (16/63) were analyzed for ß-trace protein. Results were compared against patient chart data for the presence of a CSF leak. Additional studies were performed to assess the stability, detection limit, and analytical specificity of the ß(2)-transferrin immunofixation test. RESULTS: The ß(2)-transferrin immunofixation test had a sensitivity of 100 % (40/40) and specificity of 71 % (12/17) for detection of CSF leaks. By comparison, the agarose gel test had a sensitivity of 87 % (35/40) and specificity of 94 % (16/17). ß-trace protein had a sensitivity of 100 % (10/10) and specificity of 86 % (5/6). Serum and saliva could be differentiated from CSF by the ß(2)-transferrin immunofixation test based on their migration patterns. However, whole blood samples appeared positive for ß(2)-transferrin at a threshold of ~ 4 g/L hemoglobin. At a cut-off of 3 mg/L, ß-trace protein was increased in 10/10 cases with documented CSF leak and in 1/6 patients without CSF leak. CONCLUSIONS: Both the new immunofixation test for ß(2)-transferrin and the ß-trace protein were effective at detecting CSF leaks. Users of the ß(2)-transferrin immunofixation test should be cautioned against interpreting samples with blood contamination.

Characterizing ability of serum potassium (K) and the serum K reference interval to flag hypokalemia or hyperkalemia as observed in plasma: A simulation study.

OBJECTIVES: Serum potassium (K) exhibits a positive shift relative to plasma K due to a variable amount of K release associated with clotting. Because of this variation, plasma K results outside of the reference interval (RI) for plasma (hypokalemia or hyperkalemia) in individual samples may not produce classification-concordant results in serum according to the serum RI. We examined this premise from a theoretical standpoint by simulation. DESIGN & METHODS: We used textbook K reference intervals for plasma (PRI = 3.4-4.5 mmol/L) and serum (SRI = 3.5-5.1 mmol/L). The difference between PRI and SRI is characterized by a normal distribution: serum K = plasma K + 0.35 ± 0.308 mmol/L. This transformation was applied by simulation to an observed patient data distribution for plasma K to generate a corresponding theoretical serum K distribution. Individual samples were tracked for comparison with respect to classification (below, within, above RI) for plasma and serum. RESULTS: Primary data were an all-comers plasma K patient distribution (n = 41,768; median = 4.1 mmol/L; 7.1% below PRI (hypokalemia); 15.5% above PRI (hyperkalemia)). Simulation to obtain the associated serum K yielded a right-shifted distribution (median = 4.4 mmol/L; 4.8% below SRI; 10.8% above SRI). Sensitivity for detection in serum (flagged below SRI) for samples originating as hypokalemic in plasma was 45.7% (specificity = 98.3%). Sensitivity for detection in serum (flagged above SRI) for samples originating as hyperkalemic in plasma was 56.6% (specificity = 97.6%). CONCLUSIONS: Simulation results indicate that serum K should best be thought of as an inferior substitute marker for plasma K. These results follow simply from the variable component of serum K compared to plasma K. Plasma should be the preferred specimen type for K assessment.

Does Diabetes Alter CSF Total Protein Levels? A Retrospective Cohort Study.

Background and Purpose: Elevation of total protein level in cerebrospinal fluid (CSF-TP) in diabetic patients is often disregarded by clinicians. However, existing studies on the topic have significant limitations, and therefore we aimed to explore the relationship between diabetes and CSF-TP in a large database of CSF samples. Methods: Retrospective review of all diagnostic lumbar punctures at the Ottawa Hospital between 1996-2016. Patients were excluded if they had elevated CSF cell counts, or a condition known to elevate CSF-TP. Multivariate linear regression modeling considered the effects of age, sex, and diabetes. Results: Among 6124 patients (746 with diabetes, 5378 without), mean CSF-TP did not differ significantly between groups (0.39 and 0.35 mmol/L, p = 0.2). When controlled for age and sex, there was no significant effect of diabetes on CSF-TP and no significant correlation between mean serum glucose and CSF-TP (R2 = 0.12). Conclusions: CSF-TP did not differ significantly between diabetic and non-diabetic groups, once the influence of age and sex was controlled. Elevated CSF-TP should be regarded as pathologic, even in the setting of diabetes.

Reproducible manuscript preparation with RMarkdown application to JMSACL and other Elsevier Journals.

INTRODUCTION: With the rising complexity of modern multimarker analytical techniques and notable scientific publication retractions required for erroneous statistical analysis, there is increasing awareness of the importance of research transparency and reproducibility. The development of mature open-source tools for literate programming in multiple langauge paradigms has made fully-reproducible authorship possible. OBJECTIVES: We describe the procedure for manuscript preparation using RMarkdown and the R statistical programming language with application to JMSACL or any other Elsevier journal. METHODS: An instructional manuscript has been prepared in the RMarkdown markup language with stepwise directions on preparing sections, subsections, lists, tables, figures and reference management in an entirely reproducible format. RESULTS: From RMarkdown code, a submission-ready PDF is generated and JMSACL-compatible LaTeX code is generated. These can be uploaded to the Editorial Manager. CONCLUSION: A completely reproducible manuscript preparation pipeline using the R and RMarkdown is described.

Continuous reference intervals for pediatric testosterone, sex hormone binding globulin and free testosterone using quantile regression.

Testosterone (T), sex hormone binding globulin (SHBG), free testosterone (FT) and bioavailable testosterone (BAT) are commonly employed tests in pediatric endocrinology and all require age-dependent reference intervals for interpretation. The common methods used to derive these reference intervals require decisions about data shape and/or age partition thresholds, which can result in sharp differences between age groups, particularly for pubescent children. Partitioning also results in a form of data loss, where data from one age-bin is completed disconnected from the adjacent age-bins. Non-parametric continuous reference intervals methods have previously been developed to avoid some of these drawbacks. These strategies use all the available data and smooth transitions between ages avoiding partitioning. However, the fitting process involves selection and adjustment of many parameters and it can be difficult to maintain a reproducible approach. Here we provide a workflow for non-parametric continuous reference intervals applied to T, FT, BAT, and SHBG using the R language quantregGrowth package. T measurements were determined by LC-MS/MS, FT and BAT were calculated, and SHBG was measured on the Roche Cobas e601. The continuous interval methodology is described in detail with code examples and illustrations for reproducibility.

Effective interventions to improve the quality of critically high point-of-care glucose meter results.

OBJECTIVES: Point-of-care testing (POCT) is testing performed outside the traditional laboratory, often at the patient bedside. In hospital settings, blood glucose is the most common POCT. Staff performing POCT are not usually laboratory trained; they are clinical staff with a primary focus on treating patients. Clinical staff find POCT quality assurance (QA) practices burdensome and are often non-compliant. In hospitals within EORLA (Eastern Ontario Regional Laboratories Association), all critically high POCT glucose results must be repeated prior to acting, according to policy. Compliance with this policy is audited regularly. DESIGN: and methods: All POCT glucose tests performed in participating sites between January and June 2018 and June and December 2019 were audited for compliance with the critical repeat policy. The discordant repeat rate was also determined for each audit period. Between January and May 2019, there were interventions aimed at improving compliance with the repeat policy. RESULTS: Compliance with the critical repeat policy increased from 30 to 57% in 2019 compared to 2018, following nursing education and implementation of notifications on the glucose meters themselves. The rate of discordant repeat results (>20% different from initial) also improved at most sites in 2019 compared to 2018. Nurses cited insufficient cleaning of patient hands prior to initial testing as the primary reason for discordant repeats. CONCLUSIONS: Operator compliance with POCT QA policies is an ongoing challenge requiring continual audit, feedback and education. A strong POCT multi-disciplinary committee with supports from senior and clinical leadership in an organization are key to improving compliance.

Mammalian inscuteable regulates spindle orientation and cell fate in the developing retina.

During mammalian neurogenesis, progenitor cells can divide with the mitotic spindle oriented parallel or perpendicular to the surface of the neuroepithelium. Perpendicular divisions are more likely to be asymmetric and generate one progenitor and one neuronal precursor. Whether the orientation of the mitotic spindle actually determines their asymmetric outcome is unclear. Here, we characterize a mammalian homolog of Inscuteable (mInsc), a key regulator of spindle orientation in Drosophila. mInsc is expressed temporally and spatially in a manner that suggests a role in orienting the mitotic spindle in the developing nervous system. Using retroviral RNAi in rat retinal explants, we show that downregulation of mInsc inhibits vertical divisions. This results in enhanced proliferation, consistent with a higher frequency of symmetric divisions generating two proliferating cells. Our results suggest that the orientation of neural progenitor divisions is important for cell fate specification in the retina and determines their symmetric or asymmetric outcome.

Age matters: Impact of data-driven CSF protein upper reference limits in Guillain-Barré syndrome.

Objective: We conducted a retrospective review of patients with a diagnosis of Guillain-Barré syndrome (GBS) to assess the diagnostic impact of applying age-adjusted upper limits for CSF total protein (CSF-TP) supported by a systematic literature review. Methods: Cases coded as GBS or inflammatory neuropathy for the period 2001-2016 at The Ottawa Hospital were reviewed. Cases were included if they met the Brighton criteria for GBS with a diagnostic certainty level 1 or 2 and had contemporaneous CSF-TP data. We excluded cases with CSF pleocytosis >50 and cases with Miller-Fisher syndrome. Age-adjusted reference limits were compared with conventional 0.45 and 0.6 g/L upper limits. Results: One hundred thirty-eight cases met the study criteria, with a mean age of 47 years. The mean interval from symptom onset to lumbar puncture was 7.9 days, and mean CSF-TP was 1.23 g/L. There was a strong correlation between rising CSF-TP and time to lumbar puncture. Age-adjusted CSF-TP had a significantly lower sensitivity of only 45% in the first week (32% in the first 3 days) compared with 70% in the first week for the 0.45 g/L limit. All upper limits gained high sensitivity after the first week. Conclusions: The low sensitivity of CSF-TP for the diagnosis of GBS is exacerbated by age-adjusted upper limits. The main role of lumbar puncture in GBS in the first week may be to help exclude other inflammatory or neoplastic etiologies of acute neuropathy. After the first week, the magnitude of the CSF-TP rise reduces the effect of different upper reference limits.