Definitions and major prerequisites of direct and indirect approaches for estimating reference limits.

Reference intervals are established either by direct or indirect approaches. Whereas the definition of direct is well established, the definition of indirect is still a matter of debate. In this paper, a general definition that covers all indirect models presently in use is proposed. With the upcoming popularity of indirect models, it has become evident that further partitioning strategies are required to minimize the risk of patients' false classifications. With indirect methods, such partitions are much easier to execute than with direct methods. The authors believe that the future of reference interval estimation belongs to indirect models with big data pools either from one laboratory or combined from several regional centres (if necessary). Independent of the approach applied, the quality assurance of the pre-analytical and analytical phase, considering biological variables and other confounding factors, is essential.

Problems with estimating reference change values (critical differences).

The concept of reference change values (RCVs) for diagnosis and monitoring of diseases has become well established. Several models habe been developed, e. g. one assuming a normal distribution and another one for a log-normal distribution. RCV values calculated for some measurands with both models are compared with each other and led to similar results. A few examples led to RCV values which are not plausible for diagnostic purposes. Although statistical concepts of RCV values are well established, their clinical relevance remains questionable at least for some measurands. Studies with clinicians are required whether RCVs are of practical usefulness.

The importance of correct stratifications when comparing directly and indirectly estimated reference intervals.

OBJECTIVES: There are generally two major reasons for the comparison of reference intervals (RIs): when externally determined RIs (from the literature or provided by a manufacturer) are compared with presently used intra-laboratory RIs and when indirectly estimated RIs are compared with directly established RIs. Discrepancies within these comparisons may occur for two reasons: 1. the pre-analytical and/or analytical conditions do not agree and/or 2. biological variables influencing the establishment of RIs have not been considered adequately. If directly and indirectly estimated reference intervals (RIs) are compared with each other, they very often agree. Sometimes, however, a comparison may differ, with the reason for any discrepancy not being further studied. A major reason for differences in the comparison of RIs is that the requirement for stratification has been neglected. METHODS: The present report outlines the consequences to RI comparison if stratification is neglected during RI determination with the main variables affecting RIs being sex and age. Alanine aminotransferase was chosen as an example in which the RIs depend on both these factors. RESULTS: Both direct and indirect approaches lead to erroneous RIs if stratification for variables which are known to affect the estimation of RIs is not performed adequately. However, failing to include a required stratification in procedures for directly determined RIs affects the outcome in a different way to indirectly determined RIs. CONCLUSIONS: The resulting difference between direct and indirect RIs is often misinterpreted as an incorrect RI estimation of the indirect method.

A new indirect estimation of reference intervals: truncated minimum chi-square (TMC) approach.

All known direct and indirect approaches for the estimation of reference intervals (RIs) have difficulties in processing very skewed data with a high percentage of values at or below the detection limit. A new model for the indirect estimation of RIs is proposed, which can be applied even to extremely skewed data distributions with a relatively high percentage of data at or below the detection limit. Furthermore, it fits better to some simulated data files than other indirect methods. The approach starts with a quantile-quantile plot providing preliminary estimates for the parameters (λ, µ, σ) of the assumed power normal distribution. These are iteratively refined by a truncated minimum chi-square (TMC) estimation. The finally estimated parameters are used to calculate the 95% reference interval. Confidence intervals for the interval limits are calculated by the asymptotic formula for quantiles, and tolerance limits are determined via bootstrapping. If age intervals are given, the procedure is applied per age interval and a spline function describes the age dependency of the reference limits by a continuous function. The approach can be performed in the statistical package R and on the Excel platform.

Next-generation reference intervals for pediatric hematology.

Background Interpreting hematology analytes in children is challenging due to the extensive changes in hematopoiesis that accompany physiological development and lead to pronounced sex- and age-specific dynamics. Continuous percentile charts from birth to adulthood allow accurate consideration of these dynamics. However, the ethical and practical challenges unique to pediatric reference intervals have restricted the creation of such percentile charts, and limitations in current approaches to laboratory test result displays restrict their use when guiding clinical decisions. Methods We employed an improved data-driven approach to create percentile charts from laboratory data collected during patient care in 10 German centers (9,576,910 samples from 358,292 patients, 412,905-1,278,987 samples per analyte). We demonstrate visualization of hematology test results using percentile charts and z-scores (www.pedref.org/hematology) and assess the potential of percentiles and z-scores to support diagnosis of different hematological diseases. Results We created percentile charts for hemoglobin, hematocrit, red cell indices, red cell count, red cell distribution width, white cell count and platelet count in girls and boys from birth to 18 years of age. Comparison of pediatricians evaluating complex clinical scenarios using percentile charts versus conventional/tabular representations shows that percentile charts can enhance physician assessment in selected example cases. Age-specific percentiles and z-scores, compared with absolute test results, improve the identification of children with blood count abnormalities and the discrimination between different hematological diseases. Conclusions The provided reference intervals enable precise assessment of pediatric hematology test results. Representation of test results using percentiles and z-scores facilitates their interpretation and demonstrates the potential of digital approaches to improve clinical decision-making.

Indirect determination of hematology reference intervals in adult patients on Beckman Coulter UniCell DxH 800 and Abbott CELL-DYN Sapphire devices.

Background Conventional establishment of reference intervals for hematological analytes is challenging due to the need to recruit healthy persons. Indirect methods address this by deriving reference intervals from clinical laboratory databases which contain large datasets of both physiological and pathological test results. Methods We used the "Reference Limit Estimator" (RLE) to establish reference intervals for common hematology analytes in adults aged 18-60 years. One hundred and ninety-five samples from 44,519 patients, measured on two different devices in a tertiary care center were analyzed. We examined the influence of patient cohorts with an increasing proportion of abnormal test results, compared sample selection strategies, explored inter-device differences, and analyzed the stability of reference intervals in simulated datasets with varying overlap of pathological and physiological test results. Results Reference intervals for hemoglobin, hematocrit, red cell count and platelet count remained stable, even if large numbers of pathological samples were included. Reference intervals for red cell indices, red cell distribution width and leukocyte count were sufficiently stable, if patient cohorts with the highest fraction of pathological samples were excluded. In simulated datasets, estimated reference limits shifted, if the pathological dataset contributed more than 15%-20% of total samples and approximated the physiological distribution. Advanced sample selection techniques did not improve the algorithm's performance. Inter-device differences were small except for red cell distribution width. Conclusions The RLE is well-suited to create reference intervals from clinical laboratory databases even in the challenging setting of a adult tertiary care center. The procedure can be used as a complement for reference interval determination where conventional approaches are limited.

Biological variables influencing the estimation of reference limits.

Reference limits (RLs) are required to evaluate laboratory results for medical decisions. The establishment of RL depends on the pre-analytical and the analytical conditions. Furthermore, biological characteristics of the sub-population chosen to provide the reference samples may influence the RL. The most important biological preconditions are gender, age, chronobiological influences, posture, regional and ethnic effects. The influence of these components varies and is often neglected. Therefore, a list of biological variables is collected from the literature and their influence on the estimation of RL is discussed. Biological preconditions must be specified if RL are reported as well for directly as for indirectly estimated RL. The influence of biological variables is especially important if RL established by direct methods are compared with those derived from indirect techniques. Even if these factors are not incorporated into the estimation of RL, their understanding can assist the interpretation of laboratory results of an individual.

Indirect methods for reference interval determination - review and recommendations.

Reference intervals are a vital part of the information supplied by clinical laboratories to support interpretation of numerical pathology results such as are produced in clinical chemistry and hematology laboratories. The traditional method for establishing reference intervals, known as the direct approach, is based on collecting samples from members of a preselected reference population, making the measurements and then determining the intervals. An alternative approach is to perform analysis of results generated as part of routine pathology testing and using appropriate statistical techniques to determine reference intervals. This is known as the indirect approach. This paper from a working group of the International Federation of Clinical Chemistry (IFCC) Committee on Reference Intervals and Decision Limits (C-RIDL) aims to summarize current thinking on indirect approaches to reference intervals. The indirect approach has some major potential advantages compared with direct methods. The processes are faster, cheaper and do not involve patient inconvenience, discomfort or the risks associated with generating new patient health information. Indirect methods also use the same preanalytical and analytical techniques used for patient management and can provide very large numbers for assessment. Limitations to the indirect methods include possible effects of diseased subpopulations on the derived interval. The IFCC C-RIDL aims to encourage the use of indirect methods to establish and verify reference intervals, to promote publication of such intervals with clear explanation of the process used and also to support the development of improved statistical techniques for these studies.

Critical comments to a recent EFLM recommendation for the review of reference intervals.

In a recent EFLM recommendation on reference intervals by Henny et al., the direct approach for determining reference intervals was proposed as the only presently accepted "gold" standard. Some essential drawbacks of the direct approach were not sufficiently emphasized, such as unacceptably wide confidence limits due to the limited number of observations claimed and the practical usability for only a limited age range. Indirect procedures avoid these disadvantages of the direct approach. Furthermore, indirect approaches are well suited for reference limits with large variations during lifetime and for common reference limits.

Pediatric reference intervals for alkaline phosphatase.

BACKGROUND: Interpretation of alkaline phosphatase activity in children is challenging due to extensive changes with growth and puberty leading to distinct sex- and age-specific dynamics. Continuous percentile charts from birth to adulthood allow accurate consideration of these dynamics and seem reasonable for an analyte as closely linked to growth as alkaline phosphatase. However, the ethical and practical challenges unique to pediatric reference intervals have restricted the creation of such percentile charts, resulting in limitations when clinical decisions are based on alkaline phosphatase activity. METHODS: We applied an indirect method to generate percentile charts for alkaline phosphatase activity using clinical laboratory data collected during the clinical care of patients. A total of 361,405 samples from 124,440 patients from six German tertiary care centers and one German laboratory service provider measured between January 2004 and June 2015 were analyzed. Measurement of alkaline phosphatase activity was performed on Roche Cobas analyzers using the IFCC's photometric method. RESULTS: We created percentile charts for alkaline phosphatase activity in girls and boys from birth to 18 years which can be used as reference intervals. Additionally, data tables of age- and sex-specific percentile values allow the incorporation of these results into laboratory information systems. CONCLUSIONS: The percentile charts provided enable the appropriate differential diagnosis of changes in alkaline phosphatase activity due to disease and changes due to physiological development. After local validation, integration of the provided percentile charts into result reporting facilitates precise assessment of alkaline phosphatase dynamics in pediatrics.

Quantity quotient reporting. Comparison of various models.

Quantity quotient (QQ) reporting has been proposed by several authors to improve or support the present situation of presenting quantitative laboratory results. This proposal is based on a concept (symmetrical model) known from the intelligence quotient, which was developed to make intelligence tests comparable. In laboratory medicine, however, most measurands follow a non-symmetrical (skewed) distribution, leading to a compression of the QQ values at the lower end of the reference interval. This effect can be avoided by several alternatives. Three models considering non-symmetrical distributions are compared with the symmetrical model in the present study. The corresponding algorithms can be easily handled on the Excel platform. Graphical presentation of the QQ allows a quick overview of test results if they occur in a large number.

Age- and sex-specific dynamics in 22 hematologic and biochemical analytes from birth to adolescence.

BACKGROUND: Pediatric laboratory test results must be interpreted in the context of interindividual variation and age- and sex-dependent dynamics. Reference intervals as presently defined for separate age groups can only approximate the age-related dynamics encountered in pediatrics. Continuous reference intervals from birth to adulthood are not available for most laboratory analytes because of the ethical and practical constraints of defining reference intervals using a population of healthy community children. We applied an indirect method to generate continuous reference intervals for 22 hematologic and biochemical analytes by analyzing clinical laboratory data from blood samples taken during clinical care of patients. METHODS: We included samples from 32 000 different inpatients and outpatients (167 000 samples per analyte) from a German pediatric tertiary care center. Measurements were performed on a Sysmex-XE 2100 and a Cobas Integra 800 during clinical care over a 6-year period. The distribution of samples considered normal was estimated with an established indirect statistical approach and used for the calculation of reference intervals. RESULTS: We provide continuous reference intervals from birth to adulthood for 9 hematology analytes (hemoglobin, hematocrit, red cell indices, red cell count, red cell distribution width, white cell count, and platelet count) and 13 biochemical analytes (sodium, chloride, potassium, calcium, magnesium, phosphate, creatinine, aspartate transaminase, alanine transaminase, γ-glutamyltransferase, alkaline phosphatase, lactate dehydrogenase, and total protein). CONCLUSIONS: Continuous reference intervals capture the population changes in laboratory analytes during pediatric development more accurately than age groups. After local validation, the reference intervals provided should allow a more precise consideration of these dynamics in clinical decision making.

Optimizing the use of the "state-of-the-art" performance criteria.

The organizers of the first EFLM Strategic Conference "Defining analytical performance goals" identified three models for defining analytical performance goals in laboratory medicine. Whereas the highest level of model 1 (outcome studies) is difficult to implement, the other levels are more or less based on subjective opinions of experts, with models 2 (based on biological variation) and 3 (defined by the state-of-the-art) being more objective. A working group of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) proposes a combination of models 2 and 3 to overcome some disadvantages inherent to both models. In the new model, the permissible imprecision is not defined as a constant proportion of biological variation but by a non-linear relationship between permissible analytical and biological variation. Furthermore, the permissible imprecision is referred to the target quantity value. The biological variation is derived from the reference interval, if appropriate, after logarithmic transformation of the reference limits.

Permissible limits for uncertainty of measurement in laboratory medicine.

The international standard ISO 15189 requires that medical laboratories estimate the uncertainty of their quantitative test results obtained from patients' specimens. The standard does not provide details how and within which limits the measurement uncertainty should be determined. The most common concept for establishing permissible uncertainty limits is to relate them on biological variation defining the rate of false positive results or to base the limits on the state-of-the-art. The state-of-the-art is usually derived from data provided by a group of selected medical laboratories. The approach on biological variation should be preferred because of its transparency and scientific base. Hitherto, all recommendations were based on a linear relationship between biological and analytical variation leading to limits which are sometimes too stringent or too permissive for routine testing in laboratory medicine. In contrast, the present proposal is based on a non-linear relationship between biological and analytical variation leading to more realistic limits. The proposed algorithms can be applied to all measurands and consider any quantity to be assured. The suggested approach tries to provide the above mentioned details and is a compromise between the biological variation concept, the GUM uncertainty model and the technical state-of-the-art.

Indirect determination of pediatric blood count reference intervals.

BACKGROUND: Determination of pediatric reference intervals (RIs) for laboratory quantities, including hematological quantities, is complex. The measured quantities vary by age, and obtaining samples from healthy children is difficult. Many widely used RIs are derived from small sample numbers and are split into arbitrary discrete age intervals. Use of intra-laboratory RIs specific to the examined population and analytical device used is not yet fully established. Indirect methods address these issues by deriving RIs from clinical laboratory databases which contain large datasets of both healthy and pathological samples. METHODS: A refined indirect approach was used to create continuous age-dependent RIs for blood count quantities and sodium from birth to adulthood. The dataset for each quantity consisted of 60,000 individual samples from our clinical laboratory. Patient samples were separated according to age, and a density function of the proportion of healthy samples was estimated for each age group. The resulting RIs were merged to obtain continuous RIs from birth to adulthood. RESULTS: The obtained RIs were compared to RIs generated by identical laboratory instruments, and to population-specific RIs created using conventional methods. This comparison showed a high concordance of reference limits and their age-dependent dynamics. CONCLUSIONS: The indirect approach reported here is well-suited to create continuous, intra-laboratory RIs from clinical laboratory databases and showed that the RIs generated are comparable to those created using established methods. The procedure can be transferred to other laboratory quantities and can be used as an alternative method for RI determination where conventional approaches are limited.

A pragmatic proposal for permissible limits in external quality assessment schemes with a compromise between biological variation and the state of the art.

Permissible limits for internal and external quality assurance are either based on biological variation or on the state of the art (technical feasibility). The former approach has a scientific basis, but, in some cases, leads to limits which are either not achievable under the present technology, or which are not stringent enough. If proficiency testing is mandatory, stringent limits which cannot be fulfilled by the majority of laboratories could lead to juristic consequences. Therefore, most national guidelines were based on the state of the art, however, without providing the underlying reasoning. A simple algorithm for permissible limits in external quality assessment schemes (EQAS) is proposed based on biological variation, technical feasibility and correlated to the rate of false positive results. The proposed limits are compared with some limits from several EQAS (RiliBÄK, SEKK, RCPA, CLIA, PROLARIT). The suggested limits are slightly more stringent than the German RiliBÄK, less stringent than the Australasian guidelines and agreed best with the Czech SEKK and the Italian PROLARIT scheme. The graphical presentation of permissible limits strictly derived of biological variation with the proposed limits led to straight lines with different slopes and a cross-over at the limits for quantities with a medium biological variation (e.g., trijodthyronine). The greatest discordance between the various recommendations was observed for calcium, chloride, hemoglobin A(1c) and sodium.

Permissible performance limits of regression analyses in method comparisons.

Method comparisons are indispensable tools for the extensive validation of analytic procedures. Laboratories often only want to know whether an established procedure (x-method) can be replaced by another one (y-method) without interfering with diagnostic purposes. Then split patients' samples are analyzed more or less simultaneously with both procedures designed to measure the same quantity. The measured values are usually presented graphically as a scatter or difference plots. The two methods are considered to be equivalent (comparable) if the data pairs scatter around the line of equality (x=y line) within permissible equivalence lines. It is proposed to derive these limits of permissible imprecision limits which are based on false-positive error rates. If all data pairs are within the limits, both methods lead to comparable false error rates. If one or more data pairs are outside the permissible equivalence limits, the x-method cannot simply be replaced by the y-method and further studies are required. The discordance may be caused either by aberrant values (outliers), non-linearity, bias or a higher variation of e.g., the y-values. The spread around the line of best fit can detect possible interferences if more than 1% of the data pairs are outside permissible spread lines in a scatter plot. Because bias between methods and imprecision can be inter-related, both require specific examinations for their identification.

A new concept to derive permissible limits for analytical imprecision and bias considering diagnostic requirements and technical state-of-the-art.

BACKGROUND: Permissible limits of analytical imprecision and bias are usually derived either from biological variability or from the state of the art. Both concepts require information from external sources which often lack transparency and are difficult to integrate in medical decision-making. Additionally, physicians may be interested in knowing the probability of decision errors due to analytical uncertainty. Therefore, an approach was developed which combines all three concepts. METHODS: The empirical (observed) biological variation was derived from reference ranges used by the laboratory (CV(E)). CV(E) was corrected to get the biological variation in the theoretical absence of analytical imprecision (CV(C)). Relatively simple equations were derived from the relationship between biological variation and the analytical imprecision (CV(A)) to calculate permissible imprecision and bias. Five quality classes are proposed for the various analytes reflecting the false-positive error rates (FPR). These classes characterize analytical procedures according to their theoretical specificity (FPR). Thus, the new approach combines the theoretical base of biological variation with the technical state-of-the-art. RESULTS AND CONCLUSIONS: As practical examples, the permissible imprecision and bias limits were estimated for a selection of quantities. The limits found were more realistic than present proposals based on Cotlove's rule (fixed fraction of biological variation), but slightly more stringent than national consensus values based on the state-of-the-art. Imprecision and bias do not affect FPR equally, and, therefore, should be assessed separately. It is proposed to insert monthly imprecision and bias results calculated after each control cycle in a table with five quality classes. This table provides a simple overview of the analytical quality performance of the entire laboratory with one glance and can be handled on the Excel platform.