A pipeline for the fully automated estimation of continuous reference intervals using real-world data.

Reference intervals are essential for interpreting laboratory test results. Continuous reference intervals precisely capture physiological age-specific dynamics that occur throughout life, and thus have the potential to improve clinical decision-making. However, established approaches for estimating continuous reference intervals require samples from healthy individuals, and are therefore substantially restricted. Indirect methods operating on routine measurements enable the estimation of one-dimensional reference intervals, however, no automated approach exists that integrates the dependency on a continuous covariate like age. We propose an integrated pipeline for the fully automated estimation of continuous reference intervals expressed as a generalized additive model for location, scale and shape based on discrete model estimates using an indirect method (refineR). The results are free of subjective user-input, enable conversion of test results into z-scores and can be integrated into laboratory information systems. Comparison of our results to established and validated reference intervals from the CALIPER and PEDREF studies and manufacturers' package inserts shows good agreement of reference limits, indicating that the proposed pipeline generates high-quality results. In conclusion, the developed pipeline enables the generation of high-precision percentile charts and continuous reference intervals. It represents the first parameter-less and fully automated solution for the indirect estimation of continuous reference intervals.

Pharmacokinetic-pharmacodynamic modeling of maintenance therapy for childhood acute lymphoblastic leukemia.

In the treatment of childhood acute lymphoblastic leukemia (ALL), current protocols combine initial high-dose multiagent chemotherapy with prolonged oral therapy with 6-mercaptopurine (6MP) and low-dose methotrexate (MTX) maintenance therapy. Decades of research on ALL treatment have resulted in survival rates of approximately 90%. However, dose-response relationships vary widely between patients and insight into the influencing factors, that would allow for improved personalized treatment management, is insufficient. We use a detailed data set with measurements of thioguanine nucleotides and MTX in red blood cells and absolute neutrophil count (ANC) to develop pharmacokinetic models for 6MP and MTX, as well as a pharmacokinetic-pharmacodynamic (PKPD) model capable of predicting individual ANC levels and thus contributing to the development of personalized treatment strategies. Here, we show that integrating metabolite measurements in red blood cells into the full PKPD model improves results when less data is available, but that model predictions are comparable to those of a fixed pharmacokinetic model when data availability is not limited, providing further evidence of the quality of existing models. With this comprehensive model development leading to dynamics similar to simpler models, we validate the suitability of this model structure and provide a foundation for further exploration of maintenance therapy strategies through simulation and optimization.

Temporal evolution and differential patterns of cellular reconstitution after therapy for childhood cancers.

The cellular reconstitution after childhood cancer therapy is associated with the risk of infection and efficacy of revaccination. Many studies have described the reconstitution after stem cell transplantation (SCT). The recovery after cancer treatment in children who have not undergone SCT has mainly been investigated in acute lymphoblastic leukemia (ALL), less for solid tumors. Here, we have examined the temporal evolution of total leukocyte, neutrophil and lymphocyte counts as surrogate parameters for the post-therapeutic immune recovery in a cohort of n = 52 patients with ALL in comparison to n = 58 patients with Hodgkin's disease (HD) and n = 22 patients with Ewing sarcoma (ES). Patients with ALL showed an efficient increase in blood counts reaching the age-adjusted lower limits of normal between 4 and 5 months after the end of maintenance therapy. The two groups of patients with HD and ES exhibited a comparably delayed recovery of total leukocytes due to a protracted post-therapeutic lymphopenia which was most pronounced in patients with HD after irradiation. Overall, we observed a clearly more efficient resurgence of total lymphocyte counts in patients aged below 12 years compared to patients aged 12 to 18 years. Our results underline that the kinetics of cellular reconstitution after therapy for HD and ES differ significantly from ALL and depend on treatment regimens and modalities as well as on patient age. This suggests a need for disease, treatment, and age specific recommendations concerning the duration of infection prophylaxis and the timing of revaccination.

RIbench: A Proposed Benchmark for the Standardized Evaluation of Indirect Methods for Reference Interval Estimation.

BACKGROUND: Indirect methods leverage real-world data for the estimation of reference intervals. These constitute an active field of research, and several methods have been developed recently. So far, no standardized tool for evaluation and comparison of indirect methods exists. METHODS: We provide RIbench, a benchmarking suite for quantitative evaluation of any existing or novel indirect method. The benchmark contains simulated test sets for 10 biomarkers mimicking routine measurements of a mixed distribution of non-pathological (reference) values and pathological values. The non-pathological distributions represent 4 common distribution types: normal, skewed, heavily skewed, and skewed-and-shifted. To identify strengths and weaknesses of indirect methods, test sets have varying sample sizes and pathological distributions differ in location, extent of overlap, and fraction. For performance evaluation, we use an overall benchmark score and sub-scores derived from absolute z-score deviations between estimated and true reference limits. We illustrate the application of RIbench by evaluating and comparing the Hoffmann method and 4 modern indirect methods -TML (Truncated-Maximum-Likelihood), kosmic, TMC (Truncated-Minimum-Chi-Square), and refineR- against one another and against a nonparametric direct method (n = 120). RESULTS: For the modern indirect methods, pathological fraction and sample size had a strong influence on the results: With a pathological fraction up to 20% and a minimum sample size of 5000, most methods achieved results comparable or superior to the direct method. CONCLUSIONS: We present RIbench, an open-source R-package, for the systematic evaluation of existing and novel indirect methods. RIbench can serve as a tool for enhancement of indirect methods, improving the estimation of reference intervals.

Reference Intervals of Serum TSH from Mixed Distributions Using Truncation Points and the Kolmogorov-Smirnov Distance.

BACKGROUND: Serum TSH reference intervals (RIs) are methodology, population, and age specific. However, the ethical and practical challenges restrict the establishment of pediatric RIs using conventional approaches and advocates the use of indirect data mining-based algorithms. This study was carried out to estimate the reference interval of neonatal serum TSH in Pakistani population using an indirect approach. METHODS: A data mining of serum TSH results of neonates (≤ 1 month of age) from 2013 - 2018 was done. Two subgroups on the basis of age from birth to 5 days and 6 - 30 days were assessed. The German study group's pre-validated indirect algorithm 'KOSMIC' was utilized for the statistical analysis. RESULTS: A total of non-duplicate 82,299 neonatal serum TSH tests were retrieved over a period of 6 years, including 88% (n = 70,788) aged 0 - 5 days and 12% (n = 11,511) ranging from 6 days to 1 month. The estimated RIs for the first age partition was 0.7 (90% CI 0.6 - 0.8) to 15.5 (90% CI 12.9 - 16.2) and for the second group 0.7 (90% CI 0.5 - 0.9) to 7.8 (90% CI 6.1 - 9.9) µIU/mL. CONCLUSIONS: This study revealed age related trends in serum TSH. The study advocates the need for population specific RIs owing to the significant variations noted on comparison with previously published literature. Precise RIs become vital particularly when serum TSH is undertaken as a confirmatory test for presumptive positive results on newborn screening for congenital hypothyroidism.

Mixture density networks for the indirect estimation of reference intervals.

BACKGROUND: Reference intervals represent the expected range of physiological test results in a healthy population and are essential to support medical decision making. Particularly in the context of pediatric reference intervals, where recruitment regulations make prospective studies challenging to conduct, indirect estimation strategies are becoming increasingly important. Established indirect methods enable robust identification of the distribution of "healthy" samples from laboratory databases, which include unlabeled pathologic cases, but are currently severely limited when adjusting for essential patient characteristics such as age. Here, we propose the use of mixture density networks (MDN) to overcome this problem and model all parameters of the mixture distribution in a single step. RESULTS: Estimated reference intervals from varying settings with simulated data demonstrate the ability to accurately estimate latent distributions from unlabeled data using different implementations of MDNs. Comparing the performance with alternative estimation approaches further highlights the importance of modeling the mixture component weights as a function of the input in order to avoid biased estimates for all other parameters and the resulting reference intervals. We also provide a strategy to generate partially customized starting weights to improve proper identification of the latent components. Finally, the application on real-world hemoglobin samples provides results in line with current gold standard approaches, but also suggests further investigations with respect to adequate regularization strategies in order to prevent overfitting the data. CONCLUSIONS: Mixture density networks provide a promising approach capable of extracting the distribution of healthy samples from unlabeled laboratory databases while simultaneously and explicitly estimating all parameters and component weights as non-linear functions of the covariate(s), thereby allowing the estimation of age-dependent reference intervals in a single step. Further studies on model regularization and asymmetric component distributions are warranted to consolidate our findings and expand the scope of applications.

Differences between capillary and venous blood counts in children-A data mining approach.

BACKGROUND: Capillary sampling of blood counts is a well-established alternative to venipuncture in paediatrics. However, the sampling method has to be considered when interpreting test results, as measurements differ. Ethical and practical considerations prevent simultaneous venous and capillary sample acquisition in comprehensive paediatric cohorts that span all ages for the purpose of a direct method comparison, resulting in uncertainty regarding the interpretation of capillary test results. METHODS: We applied a data mining method to calculate the differences between capillary and venous blood count analytes using laboratory data collected during patient care. We examined 486 401 blood counts performed between 2010 and 2017 in two German paediatric tertiary care centers in children from birth to 18 years analysed on SYSMEX XE-2100 and SYSMEX XE-5000 devices, and analysed the differences between capillary and venous test results in 15 218 paired samples performed within 24 h. RESULTS: We identified the mean systematic differences between capillary and venous (capillary-venous) test results for haemoglobin (+6.5 g/L), haematocrit (+2.38%), platelet count (-7.01 × 109 /l), red cell count (+0.18 × 1012 /L), white cell count (-0.64 × 109 /L), mean corpuscular cell volume (+2.07 fl), mean corpuscular haemoglobin (+0.33 pg), mean corpuscular haemoglobin concentration (-4.4 g/L) and red cell distribution width (+0.40%). The effect of age on these mean deltas is negligible, while the levels of test results influence the difference between capillary and venous test results in most analytes. CONCLUSIONS: Our results improve guidance regarding the interpretation of capillary test results for children of all ages and in both physiological and pathological ranges.

Age- and sex-specific pediatric reference intervals for neutrophil-to-lymphocyte ratio, lymphocyte-to-monocyte ratio, and platelet-to-lymphocyte ratio.

INTRODUCTION: Neutrophil-to-lymphocyte ratio (NLR), platelet-to-lymphocyte ratio (PLR), and lymphocyte-to-monocyte ratio (LMR) are emerging biomarkers for systemic inflammation and have been shown to predict morbidity and mortality for several diseases. However, lack of pediatric reference intervals (RIs) prevents their comprehensive use in patient care and medical research. MATERIAL AND METHODS: We calculated reference intervals and corresponding confidence intervals for NLR, PLR, and LMR from birth to 18 years using a data-mining approach: We analyzed 232 746 blood counts from 60 685 patients performed during patient care and excluded patients with elevated C-reactive protein and procalcitonin. Test results were separated according to age and sex, and the distribution of physiological ratios was estimated using an indirect approach (refineR). Additionally, we investigated the ratios' diagnostic benefit for different inflammatory diseases (acute appendicitis, asthma, Bell's palsy, Henoch-Schonlein purpura, and cystic fibrosis) using the newly obtained reference intervals. RESULTS: We estimated age- and sex-specific reference intervals from birth to adulthood for NLR, PLR, and LMR. Analyses in pediatric inflammatory diseases showed that PLR and LMR were poor markers to detect the examined inflammatory diseases, while NLR was significantly increased in patients with appendicitis and asthma. CONCLUSION: We provide pediatric reference intervals for NLR, PLR, and LMR to improve the interpretation of these biomarkers in children.

Influence of Turkish origin on hematology reference intervals in the German population.

Reference intervals for laboratory test results have to be appropriate for the population in which they are used to be clinically useful. While sex and age are established partitioning criteria, patients' origin also influences laboratory test results, but is not commonly considered when creating or applying reference intervals. In the German population, stratification for ethnicity is rarely performed, and no ethnicity-specific hematology reference intervals have been reported yet. In this retrospective study, we investigated whether specific reference intervals are warranted for the numerically largest group of non-German descent, individuals originating from Turkey. To this end, we analyzed 1,314,754 test results from 167,294 patients from six German centers. Using a name-based algorithm, 1.9% of patients were identified as originating from Turkey, in line with census data and the algorithm's sensitivity. Reference intervals and their confidence intervals were calculated using an indirect data mining approach, and Turkish and non-Turkish reference limits overlapped completely or partially in nearly all analytes, regardless of age and sex, and only 5/144 (3.5%) subgroups' reference limits showed no overlap. We therefore conclude that the current practice of using common reference intervals is appropriate and allows correct clinical decision-making in patients originating from Turkey.

refineR: A Novel Algorithm for Reference Interval Estimation from Real-World Data.

Reference intervals are essential for the interpretation of laboratory test results in medicine. We propose a novel indirect approach to estimate reference intervals from real-world data as an alternative to direct methods, which require samples from healthy individuals. The presented refineR algorithm separates the non-pathological distribution from the pathological distribution of observed test results using an inverse approach and identifies the model that best explains the non-pathological distribution. To evaluate its performance, we simulated test results from six common laboratory analytes with a varying location and fraction of pathological test results. Estimated reference intervals were compared to the ground truth, an alternative indirect method (kosmic), and the direct method (N = 120 and N = 400 samples). Overall, refineR achieved the lowest mean percentage error of all methods (2.77%). Analyzing the amount of reference intervals within ± 1 total error deviation from the ground truth, refineR (82.5%) was inferior to the direct method with N = 400 samples (90.1%), but outperformed kosmic (70.8%) and the direct method with N = 120 (67.4%). Additionally, reference intervals estimated from pediatric data were comparable to published direct method studies. In conclusion, the refineR algorithm enables precise estimation of reference intervals from real-world data and represents a viable complement to the direct method.

Quantification of Translocation-Specific ctDNA Provides an Integrating Parameter for Early Assessment of Treatment Response and Risk Stratification in Ewing Sarcoma.

PURPOSE: We evaluated the predictive and prognostic value of circulating tumor DNA (ctDNA) in patients with Ewing sarcoma (EWS) treated in the EWING2008 trial. EXPERIMENTAL DESIGN: Plasma samples from 102 patients with EWS enrolled in the EWING2008 trial were obtained before and during induction chemotherapy. Genomic EWSR1 fusion sequence spanning primers and probes were used for highly specific and sensitive quantification of the levels of ctDNA by digital droplet PCR. ctDNA levels were correlated to established clinical risk factors and outcome parameters. RESULTS: Pretreatment ctDNA copy numbers were correlated with event-free and overall survival. The reduction in ctDNA levels below the detection limit was observed in most cases after only two blocks of vincristine, ifosfamide, doxorubicin, and etoposide (VIDE) induction chemotherapy. The persistence of ctDNA after two VIDE blocks was a strong predictor of poor outcomes. ctDNA levels correlated well with most established clinical risk factors; an inverse correlation was found only for the histologic response to induction therapy. ctDNA levels did not provide simple representations of tumor volume, but integrated information from various tumor characteristics represented an independent EWS tumor marker with predictive and prognostic value. CONCLUSIONS: ctDNA copy number in the plasma of patients with EWS is a quantifiable parameter for early risk stratification and can be used as a dynamic noninvasive biomarker for early prediction of treatment response and outcome of patients.

Indirect determination of serum creatinine reference intervals in a Pakistani pediatric population using big data analytics.

BACKGROUND: The indirect methods of reference intervals (RI) establishment based on data mining are utilized to overcome the ethical, practical challenges and the cost associated with the conventional direct approach. AIM: To generate RIs for serum creatinine in children and adolescents using an indirect statistical tool. METHODS: Data mining of the laboratory information system was performed for serum creatinine analyzed from birth to 17 years for both genders. The timeline was set at six years from January 2013 to December 2018. Microsoft Excel 2010 and an indirect algorithm developed by the German Society of Clinical Chemistry and Laboratory Medicine's Working Group on Guide Limits were used for the data analysis. RESULTS: Data were extracted from 96104 samples and after excluding multiple samples for the same individual, we calculated RIs for 21920 males and 14846 females, with stratification into six discrete age groups. CONCLUSION: Serum creatinine dynamics varied significantly across gender and age groups.

A Multi-User Terminology Mapping Toolbox.

Semantic interoperability is a major challenge in multi-center data sharing projects, a challenge that the German Initiative for Medical Informatics is taking up. With respect to laboratory data, enriching site-specific tests and measurements with LOINC codes appears to be a crucial step in supporting cross-institutional research. However, this effort is very time-consuming, as it requires expert knowledge of local site specifics. To ease this process, we developed a generic manual collaborative terminology mapping tool, the MIRACUM Mapper. It allows the creation of arbitrary mapping workflows involving different user roles. A mapping workflow with two user roles has been implemented at University Hospital Erlangen to support the local LOINC mapping. Additionally, the MIRACUM LabVisualizeR provides summary statistics and visualizations of analyte data. We developed a toolbox that facilitates the collaborative creation of mappings and streamlines the review as well as the validation process. The two tools are available under an open source license.

First Steps to Evaluate an NLP Tool's Medication Extraction Accuracy from Discharge Letters.

INTRODUCTION: The aim of this study is to evaluate the use of a natural language processing (NLP) software to extract medication statements from unstructured medical discharge letters. METHODS: Ten randomly selected discharge letters were extracted from the data warehouse of the University Hospital Erlangen (UHE) and manually annotated to create a gold standard. The AHD NLP tool, provided by MIRACUM's industry partner was used to annotate these discharge letters. Annotations by the NLP tool where then compared to the gold standard on two levels: phrase precision (whether or not the whole medication statement has been identified correctly) and token precision (whether or not the medication name has been identified correctly within correctly discovered medication phrases). RESULTS: The NLP tool detected medication related phrases with an overall F-measure of 0.852. The medication name has been identified correctly with an overall F-measure of 0.936. DISCUSSION: This proof-of-concept study is a first step towards an automated scalable evaluation system for MIRACUM's industry partner's NLP tool by using a gold standard. Medication phrases and names have been correctly identified in most cases by the NLP system. Future effort needs to be put into extending and validating the gold standard.

A Framework for Criteria-Based Selection and Processing of Fast Healthcare Interoperability Resources (FHIR) Data for Statistical Analysis: Design and Implementation Study.

BACKGROUND: The harmonization and standardization of digital medical information for research purposes is a challenging and ongoing collaborative effort. Current research data repositories typically require extensive efforts in harmonizing and transforming original clinical data. The Fast Healthcare Interoperability Resources (FHIR) format was designed primarily to represent clinical processes; therefore, it closely resembles the clinical data model and is more widely available across modern electronic health records. However, no common standardized data format is directly suitable for statistical analyses, and data need to be preprocessed before statistical analysis. OBJECTIVE: This study aimed to elucidate how FHIR data can be queried directly with a preprocessing service and be used for statistical analyses. METHODS: We propose that the binary JavaScript Object Notation format of the PostgreSQL (PSQL) open source database is suitable for not only storing FHIR data, but also extending it with preprocessing and filtering services, which directly transform data stored in FHIR format into prepared data subsets for statistical analysis. We specified an interface for this preprocessor, implemented and deployed it at University Hospital Erlangen-Nürnberg, generated 3 sample data sets, and analyzed the available data. RESULTS: We imported real-world patient data from 2016 to 2018 into a standard PSQL database, generating a dataset of approximately 35.5 million FHIR resources, including "Patient," "Encounter," "Condition" (diagnoses specified using International Classification of Diseases codes), "Procedure," and "Observation" (laboratory test results). We then integrated the developed preprocessing service with the PSQL database and the locally installed web-based KETOS analysis platform. Advanced statistical analyses were feasible using the developed framework using 3 clinically relevant scenarios (data-driven establishment of hemoglobin reference intervals, assessment of anemia prevalence in patients with cancer, and investigation of the adverse effects of drugs). CONCLUSIONS: This study shows how the standard open source database PSQL can be used to store FHIR data and be integrated with a specifically developed preprocessing and analysis framework. This enables dataset generation with advanced medical criteria and the integration of subsequent statistical analysis. The web-based preprocessing service can be deployed locally at the hospital level, protecting patients' privacy while being integrated with existing open source data analysis tools currently being developed across Germany.

High-resolution pediatric reference intervals for 15 biochemical analytes described using fractional polynomials.

OBJECTIVES: Assessment of children's laboratory test results requires consideration of the extensive changes that occur during physiological development and result in pronounced sex- and age-specific dynamics in many biochemical analytes. Pediatric reference intervals have to account for these dynamics, but ethical and practical challenges limit the availability of appropriate pediatric reference intervals that cover children from birth to adulthood. We have therefore initiated the multi-center data-driven PEDREF project (Next-Generation Pediatric Reference Intervals) to create pediatric reference intervals using data from laboratory information systems. METHODS: We analyzed laboratory test results from 638,683 patients (217,883-982,548 samples per analyte, a median of 603,745 test results per analyte, and 10,298,067 test results in total) performed during patient care in 13 German centers. Test results from children with repeat measurements were discarded, and we estimated the distribution of physiological test results using a validated statistical approach (kosmic). RESULTS: We report continuous pediatric reference intervals and percentile charts for alanine transaminase, aspartate transaminase, lactate dehydrogenase, alkaline phosphatase, γ-glutamyl-transferase, total protein, albumin, creatinine, urea, sodium, potassium, calcium, chloride, anorganic phosphate, and magnesium. Reference intervals are provided as tables and fractional polynomial functions (i.e., mathematical equations) that can be integrated into laboratory information systems. Additionally, Z-scores and percentiles enable the normalization of test results by age and sex to facilitate their interpretation across age groups. CONCLUSIONS: The provided reference intervals and percentile charts enable precise assessment of laboratory test results in children from birth to adulthood. Our findings highlight the pronounced dynamics in many biochemical analytes in neonates, which require particular consideration in reference intervals to support clinical decision making most effectively.

Latent class distributional regression for the estimation of non-linear reference limits from contaminated data sources.

BACKGROUND: Medical decision making based on quantitative test results depends on reliable reference intervals, which represent the range of physiological test results in a healthy population. Current methods for the estimation of reference limits focus either on modelling the age-dependent dynamics of different analytes directly in a prospective setting or the extraction of independent distributions from contaminated data sources, e.g. data with latent heterogeneity due to unlabeled pathologic cases. In this article, we propose a new method to estimate indirect reference limits with non-linear dependencies on covariates from contaminated datasets by combining the framework of mixture models and distributional regression. RESULTS: Simulation results based on mixtures of Gaussian and gamma distributions suggest accurate approximation of the true quantiles that improves with increasing sample size and decreasing overlap between the mixture components. Due to the high flexibility of the framework, initialization of the algorithm requires careful considerations regarding appropriate starting weights. Estimated quantiles from the extracted distribution of healthy hemoglobin concentration in boys and girls provide clinically useful pediatric reference limits similar to solutions obtained using different approaches which require more samples and are computationally more expensive. CONCLUSIONS: Latent class distributional regression models represent the first method to estimate indirect non-linear reference limits from a single model fit, but the general scope of applications can be extended to other scenarios with latent heterogeneity.

Model-Based Simulation of Maintenance Therapy of Childhood Acute Lymphoblastic Leukemia.

Acute lymphoblastic leukemia is the most common malignancy in childhood. Successful treatment requires initial high-intensity chemotherapy, followed by low-intensity oral maintenance therapy with oral 6-mercaptopurine (6MP) and methotrexate (MTX) until 2-3 years after disease onset. However, intra- and inter-individual variability in the pharmacokinetics (PK) and pharmacodynamics (PD) of 6MP and MTX make it challenging to balance the desired antileukemic effects with undesired excessive myelosuppression during maintenance therapy. A model to simulate the dynamics of different cell types, especially neutrophils, would be a valuable contribution to improving treatment protocols (6MP and MTX dosing regimens) and a further step to understanding the heterogeneity in treatment efficacy and toxicity. We applied and modified a recently developed semi-mechanistic PK/PD model to neutrophils and analyzed their behavior using a non-linear mixed-effects modeling approach and clinical data obtained from 116 patients. The PK model of 6MP influenced the accuracy of absolute neutrophil count (ANC) predictions, whereas the PD effect of MTX did not. Predictions based on ANC were more accurate than those based on white blood cell counts. Using the new cross-validated mathematical model, simulations of different treatment protocols showed a linear dose-effect relationship and reduced ANC variability for constant dosages. Advanced modeling allows the identification of optimized control criteria and the weighting of specific influencing factors for protocol design and individually adapted therapy to exploit the optimal effect of maintenance therapy on survival.