Impact of regression modeling on the assessment and harmonization of a point-of-care analyzer and commercial laboratory analyzer for feline plasma biochemistry testing.

BACKGROUND: Regression describes the relationship of results from two analyzers, and the generated equation can be used to harmonize results. Point-of-care (POC) analyzers cannot be calibrated by the end user, so regression offers an opportunity for calculated harmonization. Harmonization (uniformity) of laboratory results facilitates the use of common reference intervals and medical decision thresholds. OBJECTIVE: Our aims were to characterize the relationship of results for multiple biochemistry analytes on a POC and a commercial laboratory analyzer (CL) with three regression techniques and to use regression equations to harmonize the POC results with those of the CL. Harmonized results were assessed by recognized quality goals. We used harmonized results to assess the regression techniques. METHODS: After analyzer imprecision assessments, paired clinical samples were assessed with one dataset to calculate regression parameters that were applied to a second dataset. Three regression techniques were performed, and each was used to harmonize the POC results with those from the CL. POC results were assessed for bias and the number of results reaching quality goals before and after harmonization. RESULTS: All regression techniques could be used to harmonize most analytes so that 95% of results were within ASVCP TEa guidelines. Harmonization could be further improved with alternate regression techniques or exclusions. CONCLUSIONS: Regression offers a means to harmonize POC and CL analyzers. Further work is needed to assess how few samples can reliably be used and to assess likely species differences. No regression technique reliably describes the relationship between methods when correlation is poor.

Analytical performance of feline plasma on current Heska and IDEXX point-of-care biochemistry analyzers compared with a commercial laboratory analyzer.

BACKGROUND: Point-of-care (POC) biochemistry analyzers are widely used in small animal clinical practice but infrequently independently assessed for performance. OBJECTIVE: To assess the performance of two current model point-of-care biochemistry analyzers (Heska Element DC and IDEXX Catalyst) compared with a commercial laboratory analyzer (Cobas 8000). METHODS: One hundred twenty-one cats from a feline hospital population were sampled with plasma results from a single lithium heparin tube assessed on all three analyzers. Plasma biochemistry results from each POC analyzer were compared with the commercial laboratory analyzer using Bland-Altman difference plots and by determining whether the limits of agreement (LOAs) (95% of differences) fell within various quality goals after correcting for inherent bias. RESULTS: Only 7 of 14 analytes on the Heska analyzer and 2 analytes on the IDEXX analyzer attained the most stringent LOA quality goal, which was being within desirable total error based on biologic variation (TEdes). The number of analytes achieving quality goals increased with less stringent standards such as American Society of Veterinary Clinical Pathologists allowable total error (ASVCP TEA) guidelines or if <95% of clinical comparisons reaching these quality goals is considered acceptable. Widespread bias was found between both POC analyzers and the commercial laboratory analyzer. CONCLUSIONS: The performance of both POC biochemistry analyzers was variable compared with a commercial laboratory analyzer. Performance goals were only able to be attained after the bias for each analyzer was accounted for by offsetting the LOAs and quality goals set by the mean bias for each analyte on each analyzer.

The importance of quality control validation and relationships with total error quality goals and bias in the interpretation of laboratory results.

The objective of a quality system is to provide accurate and reliable results for clinical decision-making. One part of this is Quality Control (QC) validation. QC validation is not routinely applied in veterinary laboratories. This leads to the inappropriate usage of random QC rules without knowing the Probability of error detection (Ped ) and Probability of false rejection (Pfr ) of a method. In this paper, we will discuss why QC validation is important, when it should be undertaken, why QC validation is done, and why it is not commonly done. We will present the role of total analytical error (TEa) in the QC validation process and the challenges when a consensus TEa has not been published. Finally, we will also discuss the possibilities of 'gray zone' determinations and mention the effects of bias on the quality of results. Reasons for the low prevalence of performing QC validation may include (a) lack of familiarity with the concept, (b) lack of time and resources needed to conduct QC validation, and (c) lack of TEa goal for some measurands. If no TEa is available, the user may elect to use a 'reverse approach' to QC validation. This uses the CV and bias generated from the evaluation of QC measurements, specifying Ped , Pfr , and N (number of QC measurements/run). This identifies the lowest total error that can be controlled under these defined conditions, thus enabling the laboratory to have an estimate of the 'gray zone' associated with results generated with a specific assay.

Repeat patient testing quality control (RPT-QC): Background and theory.

Repeat-patient testing quality control (RPT-QC) is a version of statistical quality control (SQC) in which individual patient samples, rather than commercial control materials, are used. Whereas conventional SQC assumes control material stability and repeatedly measures the same lot of control material over time, RPT-QC uses a unique patient sample for each QC event and exploits the labile nature of patient samples under prescribed storage conditions for QC purposes. Advantages of RPT-QC include commutability, lower cost, and QC at concentrations of medical interest. Challenges include sample procurement and the establishment of control limits. The objective of this review is to compare and contrast the principles and procedures of RPT-QC and conventional SQC and to provide an overview of RPT-QC control limit establishment.

Guidelines for resident training in veterinary clinical pathology. IV: Laboratory quality management-Teaching domains, competencies, and suggested learning outcomes.

BACKGROUND: The 2019 ASVCP Education Committee Forum for Discussion, presented at the annual ASVCP/ACVP meeting, identified a need to develop recommendations for teaching laboratory quality management principles in veterinary clinical pathology residency training programs. OBJECTIVES: To present a competency-based framework for teaching laboratory quality management principles in veterinary clinical pathology residency training programs, including entrustable professional activities (EPAs), domains of competence, individual competencies, and learning outcomes. METHODS: A joint subcommittee of the ASVCP Quality Assurance and Laboratory Standards (QALS) and Education Committees executed this project. A draft guideline version was reviewed by the ASVCP membership and shared with selected ACVP committees in early 2022, and a final version was voted upon by the full QALS and Education Committees in late 2022. RESULTS: Eleven domains of competence with relevant individual competencies were identified. In addition, suggested learning outcomes and resource lists were developed. Domains and individual competencies were mapped to six EPAs. CONCLUSIONS: This guideline presents a framework for teaching principles of laboratory quality management in veterinary clinical pathology residency training programs and was designed to be comprehensive yet practical. Guidance on pedagogical terms and possible routes of implementation are included. Recommendations herein aim to improve and support resident training but may require gradual implementation, as programs phase in necessary expertise and resources. Future directions include the development of learning milestones and assessments and consideration of how recommendations intersect with the American College of Veterinary Pathologists training program accreditation and certifying examination.

Preferences of clinical pathologists for probability-modifying terms describing cytologic sample evaluations: A survey study.

BACKGROUND: A recent study identified 7 probability ranges used by clinical pathologists and associated qualitative terms used in cytology reports. Clinicians and clinical pathologists agreed that limiting the number of terms could help enhance communication between clinical pathologists and clinicians. However, the preferred terms for each range remain undetermined. OBJECTIVE: We sought to determine a single term for each probability range that could be adopted by the global veterinary clinical pathology community. METHOD: Clinical pathologists responded to a survey invitation distributed via the specialty listserv. Clinical pathologists were asked to rank previously identified terms for each probability range from "most preferred" to "least preferred." An alternative term could be proposed if they preferred a term not included in the question. The preferences were summed by rank. Where first choice ranks were within 20% of each other, the 1st and 2nd choices were added. The term with the highest counts was chosen to represent the probability range. RESULTS: The highest-ranking terms corresponding to the probability ranges of 0%-20%, 20%-50%, 50%-65%, 65%-75%, 75%-85%, 85%-95%, and 95%-100% were "no evidence for," "cannot rule out," "possible," "suspicious for," "most likely," "most consistent with," and no modifier, respectively. CONCLUSIONS: We have sampled clinical pathologists across the globe to rank terms in cytology reports associated with previously identified probability ranges to identify single qualitative terms for which there was the most agreement between clinicians and clinical pathologists. Our study provides the foundation for standardizing and limiting probability-modifying terms to improve communication with clinicians.

Veterinary clinicians prefer template-style reports with personal confidence estimates for cytologic sample evaluations.

OBJECTIVE: To examine preferences of veterinary clinical pathologists, clinicians, and students for cytology report formats. SAMPLE: 24 clinical pathologists, 1,014 veterinarians, and 93 veterinary students who were members of the Veterinary Information Network. METHODS: Members of the Veterinary Information Network responded to an online survey invitation, made available between July 11, 2023, and July 24, 2023. Respondents were randomly directed to 1 of 4 sets of cytology reports, each containing a traditional narrative format, narrative format with terms expressing a degree of confidence and associated numerical ranges, and template format with similar estimates of confidence. Respondents ranked the reports in order of preference and then provided comments about their top-ranked choice. Responses were analyzed mostly with descriptive statistics or comparisons of proportions. RESULTS: 14 of 24 clinical pathologists preferred the traditional narrative format, whereas 449 of 1,042 veterinary clinicians and veterinary students preferred the template format. Respondents (460/1,131) ranked the template format as most preferred, but the narrative format with terms expressing a degree of confidence ranked highest overall. Many respondents appeared to misunderstand the degree of confidence estimates being expressed numerically. Respondents choosing each format often stated that their preferred choice was "easiest to understand" and "most comprehensive." CLINICAL RELEVANCE: Given the preferences of veterinary clinicians and veterinary students for a template format, clinical pathologists should consider modifying the way they report evaluations of cytologic specimens. Template formats should help standardize reporting of cytologic specimens, thereby improving communication between clinical pathologists and clinicians. However, both clinicians and clinical pathologists need to better understand the purpose of terminology expressing degrees of confidence in such reports.

Repeat patient Testing-Quality control: An example of implementation in a network of commercial laboratories.

The theory and calculations underpinning Repeat Patient Testing-Quality Control (RPT-QC) have been presented in prior publications. This paper gives an example of the process used for implementing RPT-QC in a network of veterinary commercial reference laboratories and the stages associated with the transition to the sole use of RPT-QC. To employ RPT-QC in this commercial laboratory network, eight stages of implementation were identified: (1) education, (2) data collection, (3) calculations, (4) QC recording and documentation, (5) running RPT-QC in parallel with a commercially available quality control material (QCM), (6) development of a Standard Operating Procedure (SOP), (7) development of complementary aspects supporting RPT-QC, and (8) sole use of RPT-QC. Advantages of RPT-QC included cost savings for QCM and External Quality Assessment (EQA) participation and the ability to use commutable specimens with a veterinary matrix at a result level that is of clinical significance for the species. A disadvantage of RPT-QC using a single level of control was the inability to demonstrate stable performance over a range of results. Future avenues for investigation include ongoing refinement of control limits using a pooled standard deviation of the duplicates (SDdup), Sdup over time, investigation of blood samples from species other than the dog, and manipulation of specimens to produce "low abnormal" or "high abnormal" RPT-QC specimens.

Repeat patient testing-quality control with canine samples shows promise as an alternative to commercial quality control material for a network of four Sysmex XT-2000iV hematology analyzers.

BACKGROUND: Repeat patient testing-quality control (RPT-QC) uses retained patient samples as an alternative to commercial quality control material (QCM). We elected to calculate and validate RPT-QC limits for red blood cell count (RBC), hemoglobin (HBG), hematocrit (HCT), and white blood cell count (WBC). OBJECTIVES: (1) To validate RPT-QC across a network of four harmonized Sysmex XT-2000iV hematology analyzers and determine the total error that can be controlled with RPT-QC. (2) To generate quality control (QC) limits using the standard deviation (SD) of the duplicate measurement differences and determine a suitable simple QC rule with a probability of error detection >0.85 and probability of false rejection <0.05. (3) Monitor RPT-QC using sigma metrics as a performance indicator and (4) to challenge RPT-QC to ensure acceptable sensitivity. METHODS: Fresh adult canine EDTA samples with results within reference intervals were selected and run again on days 2, 3, and 4. QC limits were generated from the SD of the duplicate measurement differences. The QC limits were challenged using interventions designed to promote unstable system performance. The total error detectable by RPT-QC was determined using EZRULES 3 software. RESULTS: In all, 20-40 data points were needed for RPT-QC calculations and validated using 20 additional data points. The calculated limits differed among the network of analyzers. The total error that could be controlled was the same or better than that of the manufacturer's commercially available quality control material using the same analyzer for all measurands except hematocrit, which required a higher total error goal than that proposed by ASVCP guidelines to achieve an acceptable probability of error detection. The challenges designed to mimic unstable system performance were successfully detected as out-of-control QC. CONCLUSIONS: The challenges for RPT-QC resulted in acceptable detection of potential unstable system performance. This initial study demonstrates that RPT-QC limits differ among the network of Sysmex XT-2000iV analyzers, indicating a requirement to customize for the individual analyzer and laboratory conditions. RPT-QC could achieve ASVCP total allowable error goals for RBC, HGB, and WBC, but not for HCT. Sigma metrics were consistently >5.5 for RBC, HGB, and WBC, but not for HCT.

Clinical pathologists should limit modifier terms used to denote probability of a diagnosis: a survey-based study.

OBJECTIVE: To examine the probability estimates for modifying terms used by clinical pathologists when interpreting cytologic samples and compare these to probability estimates assigned to these terms by clinicians, and to provide restricted, standardizing terms used in cytology reports. SAMPLE: 49 clinical pathologists and 466 Veterinary Information Network members responded to 2 similar surveys. PROCEDURES: Online surveys were distributed to diplomates of the European College of Veterinary Clinical Pathologists and clinician members of the Veterinary Information Network, made available between March 17, 2022, through May 5, 2022. Respondents assigned a range of probabilities to each of 18 modifier terms used by clinical pathologists to denote probability associated with diagnoses; clinicians identified terms that would affect their treatment decisions in cases of canine lymphoma. Respondents then provided thoughts about restricting and standardizing modifying terms and assigning numeric estimates in reports. RESULTS: 49 clinical pathologists and 466 clinicians provided responses. For many terms, probability ranges agreed between the 2 groups. However, differences in estimated probability inferred by a term existed for at least 6 terms. Modifying terms could be restricted to 7 largely nonoverlapping terms that spanned the range of probabilities. Clinicians preferred having numeric estimates of probability, but clinical pathologists resisted providing such estimates in reports. CLINICAL RELEVANCE: Reducing and standardizing the number of modifying terms to reflect specific probability ranges would reduce disagreement between the clinical pathologist's intended probability range and the clinician's interpretation of a modifying term. This could result in fewer errors in interpretation and better patient care.

Recommendations to improve the patient experience and avoid bias when prenatal screening/testing.

While prenatal screening and testing have expanded substantially over the past decade and provide access to more genetic information, expectant parents are more likely to describe the diagnosis experience as negative than positive. In addition, the conversations that take place during these experiences sometimes reflect unconscious bias against people with disabilities. Consequently, an interdisciplinary committee of experts, including people with disabilities, family members, disability organization leaders, healthcare and genetics professionals, and bioethicists, reviewed selected published and gray literature comparing the current state of the administration of prenatal testing to the ideal state. Subsequently, the interdisciplinary team created recommendations for clinicians, public health agencies, medical organizations, federal agencies, and other stakeholders involved with administering prenatal screening and testing to create better patient experiences; conduct training for healthcare professionals; create, enforce, and fund policies and guidelines; and engage in more robust data collection and research efforts.

Quality control validation for a veterinary laboratory network of six Sysmex XT-2000iV hematology analyzers.

BACKGROUND: Quality control (QC) validation is an important step in the laboratory harmonization process. This includes the application of statistical QC requirements, procedures, and control rules to identify and maintain ongoing stable analytical performance. This provides confidence in the production of patient results that are suitable for clinical interpretation across a network of veterinary laboratories. OBJECTIVES: To determine that a higher probability of error detection (Ped ) and lower probability of false rejection (Pfr ) using a simple control rule and one level of quality control material (QCM) could be achieved using observed analytical performance than by using the manufacturer's acceptable ranges for QCM on the Sysmex XT-2000iV hematology analyzers for veterinary use. We also determined whether Westgard Sigma Rules could be sufficient to monitor and maintain a sufficiently high level of analytical performance to support harmonization. METHODS: EZRules3 was used to investigate candidate QC rules and determine the Ped and Pfr of manufacturer's acceptable limits and also analyzer-specific observed analytical performance for each of the six Sysmex analyzers within our laboratory system using the American Society of Veterinary Clinical Pathology (ASVCP)-recommended or internal expert opinion quality goals (expressed as total allowable error, TEa ) as the quality requirement. The internal expert quality goals were generated by consensus of the Quality, Education, Planning, and Implementation (QEPI) group comprised of five clinical pathologists and seven laboratory technicians and managers. Sigma metrics, which are a useful monitoring tool and can be used in conjunction with Westgard Sigma Rules, were also calculated. RESULTS: The QC validation using the manufacturer's acceptable limits for analyzer 1 showed only 3/10 measurands reached acceptable Ped for veterinary laboratories (>0.85). For QC validation based on observed analyzer performance, the Ped was >0.94 using a 1-2.5s QC rule for the majority of observations (57/60) across the group of analyzers at the recommended TEa . We found little variation in Pfr between manufacturer acceptable limits and individual analyzer observed performance as this is a characteristic of the rule used, not the analyzer performance. CONCLUSIONS: An improved probability of error detection and probability of false rejection using a 1-2.5s QC rule for individual analyzer QC was achieved compared with the use of the manufacturers' acceptable limits for hematology in veterinary laboratories. A validated QC rule (1-2.5s) in conjunction with sigma metrics (>5.5), desirable bias, and desirable CV based on biologic variation was successful to evaluate stable analytical performance supporting continued harmonization across the network of analyzers.

Total observed error, total allowable error, and QC rules for canine serum and urine cortisol achievable with the Immulite 2000 Xpi cortisol immunoassay.

Determining a simple quality control (QC) rule for daily performance monitoring depends on the desired total allowable error (TEa) for the measurand. When no consensus TEa exists, the classical approach of QC rule validation cannot be used. Using the results of previous canine serum and urine cortisol validation studies on the Immulite 2000 Xpi, we applied a reverse engineering approach to QC rule determination, arbitrarily imposing sigma = 5, and determining the resulting TEa for the QC material (QCM; TEaQCM) and the resulting probability of error detection (Ped) for each QC rule. For the simple QC rule 12.5S with Ped = 0.96 and probability of false rejection (Pfr) = 0.03, the associated TEaQCM were 20% and 35% for serum and 28% and 24% for urine QCM1 and QCM2. If these levels of TEaQCM are acceptable for interpretation of patient sample results, then users can internally validate the 12.5S QC rule, provided that their QCM CVs and biases are similar to ours. Otherwise, more stringent QC rules can be validated by using a lower sigma to lower the TEaQCM. With spiked samples (relevant cortisol concentrations in the veterinary patient matrix) at 38.6 and 552 nmol/L of cortisol, TEaQCM at sigma = 5 were much higher (54% and 40% for serum; 90.3% and 42.8% for urine). Spiked samples generate TEa that is probably too high to be suitable for daily QC monitoring; however, it is crucial to verify spiked sample observed total error (TEo; 26% and 18% for serum, 60% and 30% for urine) < TEaQCM, and to use spiked sample TEo for patient result interpretation. In the absence of consensus TEa for cortisol in dogs, we suggest the use of a 12.5S rule, provided that users accept the associated level of TEaQCM also as clinical TEa for results interpretation.

Evaluation of Sysmex XT-2000iV analyzer performance across a network of five veterinary laboratories using a commercially available quality control material.

BACKGROUND: Laboratory and instrument harmonization is seldom reported in the veterinary literature despite its advantages to clinical interpretation, including the use of interchangeable results and common reference intervals within a system of laboratories. OBJECTIVES: A three-step process was employed to evaluate and optimize performance and then assess the appropriateness of common reference intervals across a network of six Sysmex XT-2000iV hematology analyzers at 5 commercial veterinary laboratory sites. The aims were to discover if harmonization was feasible in veterinary hematology and which quality parameters would best identify performance deviations to ensure a harmonized status could be maintained. METHODS: The performance of 10 measurands of a commercially available quality control material (Level 2-Normal e-CHECK (XE)-Hematology Control) was evaluated during three 1-month time periods. Precision and bias were assessed with Six Sigma, American Society of Veterinary Clinical Pathology (ASVCP) total error quality goals and biologic variation (BV)-based quality goal approaches to performance measurement. RESULTS: Instrument adjustments were made to 1 analyzer twice and 3 analyzers once between evaluations to improve performance and achieve harmonization. Sigma metrics improved from 37/50 > 6 to 58/60 > 6 and to all >5 over the course of the harmonization project. BV-based quality goals for desirable bias and for laboratory systems of 0.33 × CVI (within-subject biologic variation) were more sensitive and useful for assessing performance than the ASVCP total error goals. CONCLUSIONS: Optimization and harmonization were achieved, and because BV-derived bias goals were achieved, common reference intervals could be implemented across the network of analyzers.

Validation study of canine serum cortisol measurement with the Immulite 2000 Xpi cortisol immunoassay.

We report the results of validation of canine serum cortisol determination with the Immulite 2000 Xpi cortisol immunoassay (Siemens), with characterization of precision (CV), accuracy (spiking-recovery [SR] bias), and observed total error (TEo = bias + 2CV) across the reportable range, specifically at the most common interpretation thresholds for dynamic testing. Imprecision increased at increasing rate with decreasing serum cortisol concentration and bias was low, resulting in increasing TEo with decreasing serum cortisol concentration. Inter-laboratory comparison study allowed for determination of range-based bias (RB) and average bias (AB). At 38.6 and 552 nmol/L (1.4 and 20 µg/dL), between-run CV was 10% and 7.5%, respectively, and TEo ~30% and ~20%, respectively (TEo remained similar regardless of the considered bias: SR, RB, or AB). These analytical performance parameters should be considered in the interpretation of results and for future expert consensus discussions to determine recommendations for allowable total error (TEa). Importantly, the commonly used thresholds for interpretation of results were determined ~40 y ago with different methods of measurements and computation, hence updating is desirable. Quality control material (QCM) had between-run imprecision of 4% for QCM1 and 7% for QCM2; the bias was minimal for both levels. Acceptable QC rules are heavily dependent on the desired TEa for the QCM system (TEaQCM), itself limited by the desired clinical TEa. At low TEaQCM (20-33%), almost no rules were acceptable, whereas at high TEaQCM (50%), almost all rules were acceptable; further investigation is needed to determine which TEaQCM can be guaranteed by simple QC rules.

Validation study of canine urine cortisol measurement with the Immulite 2000 Xpi cortisol immunoassay.

We report here validation of the Immulite 2000 Xpi cortisol immunoassay (Siemens; with kit lot numbers <550) for measurement of urine cortisol in dogs, with characterization of the precision (CV), accuracy (spiking-recovery [SR] bias), and observed total error (TEo = bias + 2CV) across the reportable range. Linearity assessed by simple linear regression was excellent. Imprecision, SR bias, and TEo increased markedly with decreasing urine cortisol concentration. Interlaboratory comparison studies determined range-based (RB) bias and average bias (AB). The 3 biases (SR, RB, and AB) and resulting TEo differed markedly. At 38.6 and 552 nmol/L (1.4 and 20 µg/dL), between-run CVs were 10% and 4.5%, respectively, and TEoRB were ~30% and 20%, respectively, similar to observations in serum in another validation study. These analytical performance parameters should be considered for urine cortisol:creatinine ratio (UCCR) result interpretation, given that, for any hypothetical errorless urine creatinine measurement, the error % on UCCR mirrors the error % on urine cortisol. Importantly, there is no commonly used interpretation threshold for UCCR, given that UCCR varies greatly depending on measurement methods and threshold computation. To date, there is no manufacturer-provided quality control material (QCM) with target values for urine cortisol with an Immulite; for Liquicheck QCM (Bio-Rad), between-run imprecision was ~5% for both QCM levels. Acceptable QC rules are heavily dependent on the desired total allowable error (TEa) for the QCM system, itself limited by the desired clinical TEa.