Machine Learning in Laboratory Medicine: Recommendations of the IFCC Working Group.

BACKGROUND: Machine learning (ML) has been applied to an increasing number of predictive problems in laboratory medicine, and published work to date suggests that it has tremendous potential for clinical applications. However, a number of groups have noted the potential pitfalls associated with this work, particularly if certain details of the development and validation pipelines are not carefully controlled. METHODS: To address these pitfalls and other specific challenges when applying machine learning in a laboratory medicine setting, a working group of the International Federation for Clinical Chemistry and Laboratory Medicine was convened to provide a guidance document for this domain. RESULTS: This manuscript represents consensus recommendations for best practices from that committee, with the goal of improving the quality of developed and published ML models designed for use in clinical laboratories. CONCLUSIONS: The committee believes that implementation of these best practices will improve the quality and reproducibility of machine learning utilized in laboratory medicine. SUMMARY: We have provided our consensus assessment of a number of important practices that are required to ensure that valid, reproducible machine learning (ML) models can be applied to address operational and diagnostic questions in the clinical laboratory. These practices span all phases of model development, from problem formulation through predictive implementation. Although it is not possible to exhaustively discuss every potential pitfall in ML workflows, we believe that our current guidelines capture best practices for avoiding the most common and potentially dangerous errors in this important emerging field.

Benefits, limitations, and controversies on patient-based real-time quality control (PBRTQC) and the evidence behind the practice.

BACKGROUND: In recent years, there has been renewed interest in the "old" average of normals concept, now generally referred to as moving average quality control (MA QC) or patient-based real-time quality control (PBRTQC). However, there are some controversies regarding PBRTQC which this review aims to address while also indicating the current status of PBRTQC. CONTENT: This review gives the background of certain newly described optimization and validation methods. It also indicates how QC plans incorporating PBRTQC can be designed for greater effectiveness and/or (cost) efficiency. Furthermore, it discusses controversies regarding the complexity of obtaining PBRTQC settings, the replacement of iQC, and software functionality requirements. Finally, it presents evidence of the added value and practicability of PBRTQC. OUTLOOK: Recent developments in, and availability of, simulation methods to optimize and validate laboratory-specific PBRTQC procedures have enabled medical laboratories to implement PBRTQC in their daily practice. Furthermore, these methods have made it possible to demonstrate the practicability and added value of PBRTQC by means of two prospective "clinical" studies and other investigations. Although internal QC will remain an essential part of any QC plan, applying PBRTQC can now significantly improve its performance and (cost) efficiency.

Immunosuppressant quantification in intravenous microdialysate - towards novel quasi-continuous therapeutic drug monitoring in transplanted patients.

OBJECTIVES: Therapeutic drug monitoring (TDM) plays a crucial role in personalized medicine. It helps clinicians to tailor drug dosage for optimized therapy through understanding the underlying complex pharmacokinetics and pharmacodynamics. Conventional, non-continuous TDM fails to provide real-time information, which is particularly important for the initial phase of immunosuppressant therapy, e.g., with cyclosporine (CsA) and mycophenolic acid (MPA). METHODS: We analyzed the time course over 8 h of total and free of immunosuppressive drug (CsA and MPA) concentrations measured by liquid chromatography-tandem mass spectrometry (LC-MS/MS) in 16 kidney transplant patients. Besides repeated blood sampling, intravenous microdialysis was used for continuous sampling. Free drug concentrations were determined from ultracentrifuged EDTA-plasma (UC) and compared with the drug concentrations in the respective microdialysate (µD). µDs were additionally analyzed for free CsA using a novel immunosensor chip integrated into a fluorescence detection platform. The potential of microdialysis coupled with an optical immunosensor for the TDM of immunosuppressants was assessed. RESULTS: Using LC-MS/MS, the free concentrations of CsA (fCsA) and MPA (fMPA) were detectable and the time courses of total and free CsA comparable. fCsA and fMPA and area-under-the-curves (AUCs) in µDs correlated well with those determined in UCs (r≥0.79 and r≥0.88, respectively). Moreover, fCsA in µDs measured with the immunosensor correlated clearly with those determined by LC-MS/MS (r=0.82). CONCLUSIONS: The new microdialysis-supported immunosensor allows real-time analysis of immunosuppressants and tailor-made dosing according to the AUC concept. It readily lends itself to future applications as minimally invasive and continuous near-patient TDM.

Implementation of patient-based real-time quality control.

The quest to use patient results as quality control for routine clinical chemistry testing has long been driven by issues of the unavailability and cost of suitable quality control material and the matrix effects of synthetic material. Hematology laboratories were early adopters of average of normals techniques, primarily because of the difficulty in acquiring appropriate, stable quality control material, while in the chemistry laboratories, the perceived advantages and availability of synthetic material outweighed the disadvantages. However, the increasing volume of testing in clinical chemistry plus the capability of computer systems to deal with large and complex calculations has now made the use of patient-based quality control algorithms feasible. The desire to use patient-based quality control is also driven by increasing awareness that common quality control rules and frequency of analysis may fail to detect clinically significant assay biases. The non-commutability of quality control material has also become a problem as laboratories seek to harmonize results across regions and indeed globally. This review describes the history of patient-based quality control in clinical chemistry, summarizes the various approaches that can be implemented by laboratory professionals, and discusses how patient-based quality control can be integrated with traditional quality control techniques.

Understanding Patient-Based Real-Time Quality Control Using Simulation Modeling.

BACKGROUND: Patient-based real-time quality control (PBRTQC) avoids limitations of traditional quality control methods based on the measurement of stabilized control samples. However, PBRTQC needs to be adapted to the individual laboratories with parameters such as algorithm, truncation, block size, and control limit. METHODS: In a computer simulation, biases were added to real patient results of 10 analytes with diverse properties. Different PBRTQC methods were assessed on their ability to detect these biases early. RESULTS: The simulation based on 460 000 historical patient measurements for each analyte revealed several recommendations for PBRTQC. Control limit calculation with "percentiles of daily extremes" led to effective limits and allowed specification of the percentage of days with false alarms. However, changes in measurement distribution easily increased false alarms. Box-Cox but not logarithmic transformation improved error detection. Winsorization of outlying values often led to a better performance than simple outlier removal. For medians and Harrell-Davis 50 percentile estimators (HD50s), no truncation was necessary. Block size influenced medians substantially and HD50s to a lesser extent. Conversely, a change of truncation limits affected means and exponentially moving averages more than a change of block sizes. A large spread of patient measurements impeded error detection. PBRTQC methods were not always able to detect an allowable bias within the simulated 1000 erroneous measurements. A web application was developed to estimate PBRTQC performance. CONCLUSIONS: Computer simulations can optimize PBRTQC but some parameters are generally superior and can be taken as default.

Recommendation for performance verification of patient-based real-time quality control.

Patient-based real-time quality control (PBRTQC) is a laboratory tool for monitoring the performance of the testing process. It includes well-established procedures like Bull's algorithm, average of nomals, moving median, moving average (MA) and exponentially (weighted) MAs. Following the setup and optimization processes, a key step prior to the routine implementation of PBRTQC is the verification and documentation of the performance of the PBRTQC as part of the laboratory quality system. This verification process should provide a realistic representation of the performance of the PBRTQC in the environment it is being implemented in, to allow proper risk assessment by laboratory practitioners. This document focuses on the recommendation on performance verification of PBRTQC prior to implementation.

Peritoneal magnesium elimination and its clinical relevance in peritoneal dialysis patients.

BACKGROUND: Dialysis patients are at increased risk for vascular calcification and cardiovascular disease. Emerging data suggests that magnesium might be protective for the vascular system in peritoneal dialysis (PD) patients as well. However, only limited data is available on the elimination of magnesium through PD treatment. This study aims to evaluate the peritoneal magnesium elimination characteristics in comparison to other small solutes and the influence of peritoneal transport status. MATERIALS AND METHODS: Peritoneal elimination of magnesium, blood-urea-nitrogen (BUN), and creatinine during a 4-hour peritoneal equilibration test (PET) was assessed in 30 stable PD patients. Absolute magnesium elimination was compared overall and between creatinine transport tertiles. RESULTS: Median age was 61 years, 50% of patients were male, 20% were on automated PD treatment. Serum magnesium was 0.84 mmol/L, and dialysate magnesium at the end of the PET was 0.57 mmol/L in the overall cohort and did not differ significantly between tertiles. The magnesium dialysate-to-plasma ratio was significantly different between the subgroups (lower tertile: median 0.60 (minimum 0.52, maximum 0.68) vs. middle tertile: 0.64 (0.58, 0.68) vs. upper tertile: 0.69 (0.67, 0.74), p < 0.001). The elimination per liter of dialysis fluid was also significantly different (8.6 (6.6, 10.4) vs. 9.4 (8.0, 10.5) vs. 10.6 (0.2, 11.8) mg/L, p = 0.002), as was the absolute removal during the 4-hour dwell (18.6 (15.8, 21.2) vs. 19.4 (13.4, 24.6) vs. 22.7 (19.6, 31.9) mg, p = 0.007, respectively). CONCLUSION: Peritoneal magnesium elimination is similar to small solute transport characteristics. However, the absolute differences among patients with slower and faster transport types are small. Therefore, magnesium supplementation in PD patients should be guided by serum magnesium concentrations rather than the amount of peritoneal elimination.

Users Regard Operator Training as Critical for Successful POCT Applications.

BACKGROUND: Point-of-care tests (POCT) measure analytes close to the patients and are a complementary supplement to the test menu of medical laboratories. However, the involvement of many different stakeholders makes it challenging to ensure reliable results. METHODS: In a survey, we asked experienced POCT users how they control their total POCT process and what factors they consider essential for success. Results were verified in four in-depth interviews. RESULTS: Overall, 73 German participants from various medical disciplines completed the survey. All but one participant regarded operator training as important but only half of the participants' institutions conducted operator training on a regular basis. Participants often requested e-learning, but face-to-face teaching is still preferred. Twenty-one percent of participants already used e-learning and reported mixed satisfaction. Fifty-five percent of the participants never refer to the quality management manual. Instead, 94% stated that if a POCT error arises a contact person for POCT is always available at their workplace. The majority of participants think that external and, in particular, internal quality controls are important for POCT. Only a few difficulties for performing quality control such as "temporal expenditure" and "lack of information about the importance of internal quality control" were commonly mentioned. For future developments, participants expect evolution and improvements especially with regard to "measurement quality and reliability". The answers of the experts in the in-depth interviews largely corresponded with the participants of the survey. CONCLUSIONS: The importance of operator training is well established and confirmed in this work. How to conduct this training is less certain, but the answers in this survey suggest some form of blended learning with e-learning and practical elements. The discrepancy between the high importance that guidelines and other normative documents place on written information and their low practical usage was striking.

Patient-Based Real-Time Quality Control: Review and Recommendations.

For many years the concept of patient-based quality control (QC) has been discussed and implemented in hematology laboratories; however, the techniques have not been widely implemented in clinical chemistry. This is mainly because of the complexity of this form of QC, as it needs to be optimized for each population and often for each analyte. However, the clear advantages of this form of QC, together with the ongoing realization of the shortcomings of "conventional" QC, have driven a need to provide guidance to laboratories to assist in deploying patient-based QC. This overview describes the components of a patient-based QC system (calculation algorithm, block size, truncation limits, control limits) and the relationship of these to the analyte being controlled. We also discuss the need for patient-based QC system optimization using patient data from the individual testing laboratory to reliably detect systematic errors while ensuring that there are few false alarms. The term patient-based real-time quality control covers many activities that use data from patient samples to detect analytical errors. These activities include the monitoring of patient population parameters such as the mean or median analyte value or using single within-patient changes such as the delta check. In this report, we will restrict the discussion to population-based parameters. This overview is intended to serve as a guide for the implementation of a patient-based QC system. The report does not cover the clinical evaluation of the population.

External quality assessment schemes for glucose measurements in Germany: factors for successful participation, analytical performance and medical impact.

BACKGROUND: Determination of blood glucose concentration is one of the most important measurements in clinical chemistry worldwide. Analyzers in central laboratories (CL) and point-of-care tests (POCT) are both frequently used. In Germany, regular participation in external quality assessment (EQA) schemes is mandatory for laboratories performing glucose testing. METHODS: Glucose testing data from the two German EQAs "Reference Institute for Bioanalytics" (RfB) and "INSTAND - Gesellschaft zur Förderung der Qualitätssicherung in medizinischen Laboratorien" (Instand) were analyzed from 2012 to 2016. Multivariable odds ratios (OR) for the probability to reach a "good" result were calculated. Imprecision and bias were determined and clinical risk of measurement errors estimated. RESULTS: The device employed was the most important variable required for a "good" performance in all EQAs. Additional participation in an EQA for CL automated analyzers improved performance in POCT EQAs. The reciprocal effect was less pronounced. New participants performed worse than experienced participants especially in CL EQAs. Imprecision was generally smaller for CL, but some POCT devices reached a comparable performance. Large lot-to-lot differences occurred in over 10% of analyzed cases. We propose the "bias budget" as a new metric to express the maximum allowable bias that still carries acceptable medical risk. Bias budgets were smallest and clinical risks of errors greatest in the low range of measurement 60-115 mg/dL (3.3-6.4 mmol/L) for most devices. CONCLUSIONS: EQAs help to maintain high analytical performances. They generate important data that serve as the foundation for learning and improvement in the laboratory healthcare system.

Stronger Together: Aggregated Z-values of Traditional Quality Control Measurements and Patient Medians Improve Detection of Biases.

BACKGROUND: In clinical chemistry, quality control (QC) often relies on measurements of control samples, but limitations, such as a lack of commutability, compromise the ability of such measurements to detect out-of-control situations. Medians of patient results have also been used for QC purposes, but it may be difficult to distinguish changes observed in the patient population from analytical errors. This study aims to combine traditional control measurements and patient medians for facilitating detection of biases. METHODS: The software package "rSimLab" was developed to simulate measurements of 5 analytes. Internal QC measurements and patient medians were assessed for detecting impermissible biases. Various control rules combined these parameters. A Westgard-like algorithm was evaluated and new rules that aggregate Z-values of QC parameters were proposed. RESULTS: Mathematical approximations estimated the required sample size for calculating meaningful patient medians. The appropriate number was highly dependent on the ratio of the spread of sample values to their center. Instead of applying a threshold to each QC parameter separately like the Westgard algorithm, the proposed aggregation of Z-values averaged these parameters. This behavior was found beneficial, as a bias could affect QC parameters unequally, resulting in differences between their Z-transformed values. In our simulations, control rules tended to outperform the simple QC parameters they combined. The inclusion of patient medians substantially improved bias detection for some analytes. CONCLUSIONS: Patient result medians can supplement traditional QC, and aggregations of Z-values are novel and beneficial tools for QC strategies to detect biases.

Evaluation of antiphospholipid antibody assays using latent class analysis to address the lack of a reference standard.

BACKGROUND: Method evaluation of new assays for the detection of antiphospholipid antibodies (aPL) such as anti-cardiolipin (aCL) or anti-ß2-glycoprotein I (aß2-GPI) is challenging, as no internationally accepted reference material is available yet. Besides a lack of standardization, unacceptable inter-laboratory comparability of established tests is regularly observed. Owing to the absence of a commonly accepted reference standard, the evaluation of two research surface plasmon resonance (SPR) biosensor assays was performed using statistical methods from latent class analysis (LCA). METHODS: aCL and aß2-GPI IgG and IgM were measured in sera from 63 antiphospholipid syndrome patients, fulfilling the Sydney criteria, and in 34 healthy controls with four commercial assays. LCA was performed on the results and sera were assigned to the antibody-positive or antibody-negative group. Sera were subsequently evaluated in the SPR assays for aCL and aß2-GPI. Optimal cutoffs and diagnostic performances of the research systems were established employing the LCA-derived gold standard. RESULTS: With area under the curve results of 0.96 and 0.89 for the detection of aCL and aß2-GPI, the research SPR assays discriminated well between antibody-positive and antibody-negative sera. Their sensitivities and specificities were comparable to the investigated commercial immunoassays. CONCLUSIONS: SPR assays are a suitable tool for the detection of aCL and aß2-GPI with diagnostic performances not different from currently available commercial tests. LCA enabled the calculation of sensitivities and specificities for aPL assays in absence of a reference standard.

Development and alignment of undergraduate medical curricula in a web-based, dynamic Learning Opportunities, Objectives and Outcome Platform (LOOOP).

INTRODUCTION: This study presents a web-based method and its interface ensuring alignment of all parts of a curriculum map including competencies, objectives, teaching and assessment methods, workload and patient availability. Needs, acceptance and effectiveness are shown through a nine-year study. METHODS: After a comprehensive needs assessment, the curriculum map and a web-based interface "Learning Opportunities, Objectives and Outcome Platform" (LOOOP) were developed according to Harden's conceptual framework of 10-steps for curriculum mapping. The outcome was measured by surveys and results of interdisciplinary MCQ-assessments. The usage rates and functionalities were analysed. RESULTS: The implementation of LOOOP was significantly associated with improved perception of the curriculum structure by teachers and students, quality of defined objectives and their alignment with teaching and assessment, usage by students to prepare examinations and their scores in interdisciplinary MCQ-assessment. Additionally, LOOOP improved the curriculum coordination by faculty, and assisted departments for identifying patient availability for clinical training. CONCLUSION: LOOOP is well accepted among students and teachers, has positive effect on curriculum development, facilitates effective utilisation of educational resources and improves student's outcomes. Currently, LOOOP is used in five undergraduate medical curricula including 85,000 mapped learning opportunities (lectures, seminars), 5000 registered users (students, teachers) and 380,000 yearly page-visits.

How we avoid patient shortage with an integrated analysis of learning objectives and clinical data during development of undergraduate medical curricula.

Access to patients is a crucial factor for student-centred medical education. However, increasing numbers of students, teacher shortage, a patient spectrum consisting of rarer diseases, and quicker discharges limit this necessary access, and therefore pose a challenge for curriculum designers. The herein presented algorithm improves access to patients in four steps by using routinely available electronic patient data already during curriculum development. Step I: Learning objectives are mapped to appropriate ICD-10 (International Statistical Classification of Diseases) codes. Step II: It is determined which learning opportunities need to be considered first for patient allocation in order to maximise overall benefit. Step III: Hospital's departments with the highest expertise on respective learning objectives are assessed and selected for teaching. Step IV: Patients of the chosen department that present the best match for a given learning opportunity are assigned to participation. This integrated analysis of learning objectives and existing clinical data during curriculum development is a well-structured method to maximise access to patients. Furthermore, this algorithm identifies learning objectives of a curriculum that do not correspond well to the spectrum of patients of the respective teaching hospital and which should therefore be taught in learning formats without patient contact.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC).

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Preparing Laboratories for Interconnected Health Care.

In an increasingly interconnected health care system, laboratory medicine can facilitate diagnosis and treatment of patients effectively. This article describes necessary changes and points to potential challenges on a technical, content, and organizational level. As a technical precondition, electronic laboratory reports have to become machine-readable and interpretable. Terminologies such as Logical Observation Identifiers Names and Codes (LOINC), Nomenclature for Properties and Units (NPU), Unified Code for Units of Measure (UCUM), and SNOMED-CT can lead to the necessary semantic interoperability. Even if only single "atomized" results of the whole report are extracted, the necessary information for correct interpretation must be available. Therefore, interpretive comments, e.g., concerns about an increased measurement uncertainty must be electronically attached to every affected measurement result. Standardization of laboratory analyses with traceable standards and reference materials will enable knowledge transfer and safe interpretation of laboratory analyses from multiple laboratories. In an interconnected health care system, laboratories should strive to transform themselves into a data hub that not only receives samples but also extensive information about the patient. On that basis, they can return measurement results enriched with high-quality interpretive comments tailored to the individual patient and unlock the full potential of laboratory medicine.

Estimating serum-ionized magnesium concentration in hemodialysis patients.

INTRODUCTION: Cardiovascular mortality is significantly increased in kidney failure with replacement therapy (KFRT) patients, which is partly mediated by enhanced vascular calcification. Magnesium appears to have anticalcifying capabilities, and hypomagnesemia has been associated with increased mortality in KFRT patients. Ionized magnesium represents the biologically and physiologically active form. As serum ionized magnesium (Mgion ) is difficult to assess in clinical routine estimating equations derived from routinely assessed laboratory parameters could facilitate medical treatment. METHODS: We developed equations to estimate serum Mgion using linear regression analysis in 191 hemodialysis (HD) patients. Reference test was measured ionized magnesium (Mgion ). As index tests, we chose estimated Mgion using total magnesium (Mgtot ) and other laboratory and demographic variable candidates. Equations were internally validated, using 749 subsequent Mgion measurements. FINDINGS: The median patient age was 65 years, 67.5% of the patients were male. Median (interquartile range [IQR]) measured Mgion was 0.64 [0.57, 0.72] mmol/L, 11 (6%) patients were hypo- (i.e., <0.45 mmol/L) and 127 (66%) were hypermagnesemic (>0.60 mmol/L). The final equation at the end of the development process included Mgtot , serum ionized, and total calcium concentrations. In the validation dataset, bias (i.e., median difference between measured and estimated Mgion , -0.017 [-0.020, -0.014] mmol/L) and precision (i.e., IQR of bias 0.043 [0.039, 0.047] mmol/L) were small, 90% [88, 93] of estimated values were ±10% of measured values. The equation detected normomagnesemia with overall good diagnostic accuracy (area under the receiver-operating curve 0.91 [0.89, 0.93]). DISCUSSION: Mgion can be estimated from equations containing routinely assessed laboratory variables with high accuracy and good overall performance. These equations might simplify the assessment of ionized magnesium levels in the individual hemodialysis patients and help the treating physician to guide the overall treatment.

The planar cell polarity (PCP) protein Diversin translocates to the nucleus to interact with the transcription factor AF9.

The planar cell polarity (PCP) pathway, a beta-catenin-independent branch of the Wnt signaling pathway, orients cells and their appendages with respect to the body axes. Diversin, the mammalian homolog of the Drosophila PCP protein Diego, acts as a molecular switch that blocks beta-catenin-dependent and promotes beta-catenin-independent Wnt signaling. We report now that Diversin, containing several nuclear localization signals, translocates to the nucleus, where it interacts with the transcription factor AF9. Both Diversin and AF9 block canonical Wnt signaling; however, this occurs independently of each other, and does not require nuclear Diversin. In contrast, AF9 strongly augments the Diversin-driven activation of c-Jun N-terminal kinase (JNK)-dependent gene expression in the nucleus, and this augmentation largely depends on the presence of nuclear Diversin. Thus, our findings reveal that components of the PCP cascade translocate to the nucleus to participate in transcriptional regulation and PCP signaling.