Precision quality control: a dynamic model for risk-based analysis of analytical quality.

OBJECTIVES: There is continuing pressure to improve the cost effectiveness of quality control (QC) for clinical laboratory testing. Risk-based approaches are promising but recent research has uncovered problems in some common methods. There is a need for improvements in risk-based methods for quality control. METHODS: We provide an overview of a dynamic model for assay behavior. We demonstrate the practical application of the model using simulation and compare the performance of simple Shewhart QC monitoring against Westgard rules. We also demonstrate the utility of trade-off curves for analysis of QC performance. RESULTS: Westgard rules outperform simple Shewhart control over a narrow range of the trade-off curve of false-positive and false negative risk. The risk trade-off can be visualized in terms of risk, risk vs. cost, or in terms of cost. Risk trade-off curves can be "smoothed" by log transformation. CONCLUSIONS: Dynamic risk-models may provide advantages relative to static models for risk-based QC analysis.

Use of Routine Complete Blood Count Results to Rule Out Anaplasmosis Without the Need for Specific Diagnostic Testing.

BACKGROUND: Anaplasmosis presents with fever, headache, and laboratory abnormalities including leukopenia and thrombocytopenia. Polymerase chain reaction (PCR) is the preferred diagnostic but is overutilized. We determined if routine laboratory tests could exclude anaplasmosis, improving PCR utilization. METHODS: Anaplasma PCR results from a 3-year period, with associated complete blood count (CBC) and liver function test results, were retrospectively reviewed. PCR rejection criteria, based on white blood cell (WBC) and platelet (PLT) counts, were developed and prospectively applied in a mock stewardship program. If rejection criteria were met, a committee mock-refused PCR unless the patient was clinically unstable or immunocompromised. RESULTS: WBC and PLT counts were the most actionable routine tests for excluding anaplasmosis. Retrospective review demonstrated that rejection criteria of WBC ≥11 000 cells/µL or PLT ≥300 000 cells/µL would have led to PCR refusal in 428 of 1685 true-negative cases (25%) and 3 of 66 true-positive cases (5%) involving clinically unstable or immunocompromised patients. In the prospective phase, 155 of 663 PCR requests (23%) met rejection criteria and were reviewed by committee, which endorsed refusal in 110 of 155 cases (71%) and approval in 45 (29%), based on clinical criteria. PCR was negative in all 45 committee-approved cases. Only 1 of 110 mock-refused requests yielded a positive PCR result; this patient was already receiving doxycycline at the time of testing. CONCLUSIONS: A CBC-based stewardship algorithm would reduce unnecessary Anaplasma PCR testing, without missing active cases. Although the prospectively evaluated screening approach involved medical record review, this was unnecessary to prevent errors and could be replaced by a rejection comment specifying clinical situations that might warrant overriding the algorithm.

Rainbow phlebotomy collection and urine aliquots for emergency department add-on testing in the era of pandemic-driven supply shortages.

BACKGROUND: Rainbow blood draws for add-on testing in the emergency department (ED) are a common practice at our institution. We sought to determine the prevalence of this practice among reference laboratory clients and characterize the impact of pandemic-driven supply shortages. METHODS: This cross-sectional study surveyed 354 client laboratories to understand specimen collection practices in specific clinical environments and how these practices may have been affected by supply chain shortages. Data analysis by descriptive statistics was performed in Qualtrics. RESULTS: A total of 138 laboratories took the survey (39% response rate) with 57% indicating that their ED performed rainbow draws. Of these, 16% have a formal policy regarding rainbow draws, and 76% of respondents indicated that their institution was required to modify practices due to pandemic-driven supply shortages. A total of 19% indicated they routinely collect multiple urine aliquots for add-on testing. CONCLUSION: Rainbow draws and collection of urine aliquots in the ED for add-on testing are relatively common practices, with few institutions maintaining formal policies regarding the practice. Pandemic-driven supply chain shortages affected a majority of respondent laboratories and local cost-benefit analysis regarding extra specimen collection is recommended to limit waste of laboratory resources.

Artificial Intelligence Applications in Clinical Chemistry.

Artificial intelligence (AI) applications are an area of active investigation in clinical chemistry. Numerous publications have demonstrated the promise of AI across all phases of testing including preanalytic, analytic, and postanalytic phases; this includes novel methods for detecting common specimen collection errors, predicting laboratory results and diagnoses, and enhancing autoverification workflows. Although AI applications pose several ethical and operational challenges, these technologies are expected to transform the practice of the clinical chemistry laboratory in the near future.

Risk Analysis for Quality Control Part 2: Theoretical Foundations for Risk Analysis.

BACKGROUND: Risk analysis can be used to determine control limits for quality control (QC). The Parvin model is the most commonly used method for risk analysis; however; the Parvin model rests on assumptions that have been shown to produce paradoxical results and to underestimate risk. There is a need for an improved framework for risk analysis. METHODS: We developed a dynamic model (Markov Reward Model) to analyze the long-term behavior of an assay under the influence of a QC monitoring system. The model is flexible and accounts for different patterns of assay behavior (shift frequency, shift distribution) and the impact of error on patient outcomes. The model determines the distribution of undetected reported errors and the frequency of false-positive laboratory results as a function of QC settings. The model accounts for the competing risks (false detections, shifts in the mean) that cause an assay to move from an in-control state to an out-of-control state. RESULTS: The model provides a tradeoff curve that expresses the cost to prevent an unacceptable reported result in terms of laboratory cost (false-positive QC). The model can be used to optimize settings of a particular QC method or to compare the performance of different methods. CONCLUSIONS: We developed a method to evaluate that determines the cost to reduce the risk to patients (reported results with unacceptable errors) in terms of laboratory costs (false-positive QC).

Impact of assay stability on the false negative and false positive rates in quality control.

BACKGROUND: The dynamic Precision QC (PQC) model can be used to evaluate the performance of quality control (QC) monitoring systems. The model depends on inputs that describe the intrinsic shift behavior (i.e., stability) of an assay. The output of the model is a trade-off curve that shows the relationship between false negative (FN) and false positive (FP) risk events. The relationship between the inputs and outputs of this model has not yet been explored. METHODS: We used Monte Carlo simulation to generate trade-off curves using the PQC. We varied the input parameters that determine assay stability (shift probability and shift size distribution) and studied the impact of these inputs on the output (i.e., the trade-off curve relating FN risk to FP risk). RESULTS: FN risk is sensitive to the shift probability and the width of the control limits. FN risk is sensitive to the shape of the shift size distribution when the standard deviation (SD) of the shift size distribution is relatively narrow (i.e., SD < 2) but is less sensitive to the width of the shift size distribution when the SD is relatively large (i.e., SD > 2). CONCLUSIONS: Practical use of the PQC model may require the estimation of the shift probability and shift size distribution.

Risk Analysis for Quality Control Part 3: Practical Application of the Precision Quality Control Model.

BACKGROUND: We developed a theoretical framework (Precision Quality Control [PQC]) to minimize the cost of quality, but it is not known whether the method can be applied in practice. METHODS: We used data for 2 analytes, cadmium and carbohydrate-deficient transferrin (CDT), and applied the PQC framework to find the optimal control limits. These analytes were selected because they differed with respect to sigma values that are major determinants of control limits. We explored different ways to visualize the results: (a) risk trade-off (false-positive risk vs false-negative risk), (b) cost-risk trade-off (false-positive cost vs false-negative risk), and (c) cost minimization. RESULTS: We were able to use the PQC limit to produce 3 different visualizations to suggest control limits. The risk-based analysis was the simplest to apply, but the most difficult to interpret. The cost vs risk method was easy to apply but was still difficult to interpret. The cost minimization method was easy to interpret but required users to declare a willingness to pay that may be difficult to estimate. CONCLUSIONS: The PQC method can be used to find control limits that minimize the cost of quality.

Risk Analysis for Quality Control Part 1: The Impact of Transition Assumptions in the Parvin Model.

BACKGROUND: Setting quality control (QC) limits involves balancing the risk of false-positive results and false-negative results. Recent approaches to QC have focused on the assessment of false-negative results. The Parvin model is the most-used model for risk analysis. The Parvin model assumes that the system makes a transition from an in-control to an out-of-control (OOC) state but makes no further transitions after moving to the OOC state. The implications of this assumption are unclear. METHODS: We used simulation experiments to compare the performance of QC systems based on no OOC transitions allowed (NOOCTA) vs systems where OOC transitions were allowed (OOCTA). RESULTS: The NOOCTA assumption leads to paradoxical tradeoff curves between false-positive results and false-negative results. Predictions of a false-negative result based on NOOCTA were about 10 times lower than models based on OOCTA. CONCLUSIONS: The most common models for QC risk analysis underestimate false-negative results. There is a need to develop better risk-based methods for QC analysis.

Excessively low cholesterol and triglyceride levels in an apparently healthy patient.

Lipid panels are a commonly performed test in clinical laboratories. Due to the high prevalence of cardiovascular diseases around the world, it is common to see serum or plasma specimens with high results for one or more components of the lipid panel. Exceedingly low results, however, are rare and may be attributed to certain genetic, infectious, or autoimmune conditions in addition to analytical interference. Here we report a serum specimen from a 58-year-old female with cholesterol and triglyceride values below the detection limit of the assay, which was investigated to identify the cause of the anomaly. Using vitamin C test strips and high-performance liquid chromatography, the presence of high levels of antioxidant vitamin C in the patient specimen was confirmed. Subsequent treatment of the sample with the enzyme ascorbate oxidase inactivated vitamin C, leading to lipid analyte values falling within the expected range upon repeat analysis. Thus, analytical interference by vitamin C should be considered when suspiciously low lipid panel results are encountered.

Real-Time Intraoperative Consultation Reporting in the Electronic Health Record.

OBJECTIVES: We sought to make pathologists' intraoperative consultation (IOC) results immediately available to the surgical team, other clinicians, and laboratory medicine colleagues to improve communication and decrease postanalytic errors. METHODS: We created an IOC report in our stand-alone laboratory information system that could be signed out prior to, and independent of, the final report, and transfer immediately to the electronic health record (EHR) as a preliminary diagnosis. We evaluated two metrics: preliminary (IOC) result review in the EHR by clinicians and postanalytic errors. RESULTS: We assessed 2,886 IOC orders from the first 22 months after implementation. Clinicians reviewed 1,956 (68%) of the IOC results while in preliminary status, including 1,399 (48%) within the first 24 hours. We evaluated 150 cases preimplementation and 300 cases postimplementation for discrepancies between the pathologist's IOC result and the IOC result recorded by the surgeon in the operative note. Discrepancies dropped from 12 of 150 preimplementation to 6 of 150 and 7 of 150 in postimplementation years 1 and 2. One of the 25 discrepancies had a major clinical impact. CONCLUSIONS: Real-time reporting of IOC results to the EHR reliably transmits results immediately to clinical teams. This strategy reduces but does not eliminate postanalytic interpretive errors by clinical teams.

Validation and Implementation of an Ordering Alert to Improve the Efficiency of Monoclonal Gammopathy Evaluation.

OBJECTIVES: To evaluate the use of a provider ordering alert to improve laboratory efficiency and reduce costs. METHODS: We conducted a retrospective study to assess the use of an institutional reflex panel for monoclonal gammopathy evaluation. We then created a clinical decision support (CDS) alert to educate and encourage providers to change their less-efficient orders to the reflex panel. RESULTS: Our retrospective analysis demonstrated that an institutional reflex panel could be safely substituted for a less-efficient and higher-cost panel. The implemented CDS alert resulted in 79% of providers changing their high-cost order panel to an order panel based on the reflex algorithm. CONCLUSIONS: The validated decision support alert demonstrated high levels of provider acceptance and directly led to operational and cost savings within the laboratory. Furthermore, these studies highlight the value of laboratory involvement with CDS efforts to provide agile and targeted provider ordering assistance.

Decision Support Tools within the Electronic Health Record.

Laboratory tests are an integral part of the electronic health record (EHR). Providing clinical decision support (CDS) for the ordering, collection, reporting, viewing, and interpretation of laboratory testing is a fundamental function of the EHR. The implementation of a sustainable, effective laboratory CDS program requires a commitment to standardization and harmonization of the laboratory dictionaries that are the foundation of laboratory-based CDS. In this review, the authors provide an overview of the tools available within the EHR to improve decision making throughout the entire laboratory testing process, from test order to clinical action.

Order Indication Solicitation to Assess Clinical Laboratory Test Utilization: D-Dimer Order Patterns as an Illustrative Case.

BACKGROUND: A common challenge in the development of laboratory clinical decision support (CDS) and laboratory utilization management (UM) initiatives stems from the fact that many laboratory tests have multiple potential indications, limiting the ability to develop context-specific alerts. As a potential solution, we designed a CDS alert that asks the ordering clinician to provide the indication for testing, using D-dimer as an exemplar. Using data collected over a nearly 3-year period, we sought to determine whether the indication capture was a useful feature within the CDS alert and whether it provided actionable intelligence to guide the development of an UM strategy. METHODS: We extracted results and ordering data for D-dimer testing performed in our laboratory over a 35-month period. We analyzed order patterns by clinical indication, hospital service, and length of hospitalization. RESULTS: Our final data set included 13,971 result-order combinations and indeed provided actionable intelligence regarding test utilization patterns. For example, pulmonary embolism was the most common emergency department indication (86%), while disseminated intravascular coagulation was the most common inpatient indication (56%). D-dimer positivity rates increased with the duration of hospitalization and our data suggested limited utility for ordering this test in the setting of suspected venous thromboembolic disease in admitted patients. In addition, we found that D-dimer was ordered for unexpected indications including the assessment of stroke, dissection, and extracorporeal membrane oxygenation. CONCLUSIONS: Indication capture within a CDS alert and correlation with result data can provide insight into order patterns which can be used to develop future CDS strategies to guide appropriate test use by clinical indication.

Creation and Use of an Electronic Health Record Reporting Database to Improve a Laboratory Test Utilization Program.

OBJECTIVES: Laboratory-based utilization management programs typically rely primarily on data derived from the laboratory information system to analyze testing volumes for trends and utilization concerns. We wished to examine the ability of an electronic health record (EHR) laboratory orders database to improve a laboratory utilization program. METHODS: We obtained a daily file from our EHR containing data related to laboratory test ordering. We then used an automated process to import this file into a database to facilitate self-service queries and analysis. RESULTS: The EHR laboratory orders database has proven to be an important addition to our utilization management program. We provide three representative examples of how the EHR laboratory orders database has been used to address common utilization issues. We demonstrate that analysis of EHR laboratory orders data has been able to provide unique insights that cannot be obtained by review of laboratory information system data alone. Further, we provide recommendations on key EHR data fields of importance to laboratory utilization efforts. CONCLUSION: We demonstrate that an EHR laboratory orders database may be a useful tool in the monitoring and optimization of laboratory testing. We recommend that health care systems develop and maintain a database of EHR laboratory orders data and integrate this data with their laboratory utilization programs.

Career Paths of Pathology Informatics Fellowship Alumni.

BACKGROUND: The alumni of today's Pathology Informatics and Clinical Informatics fellowships fill diverse roles in academia, large health systems, and industry. The evolving training tracks and curriculum of Pathology Informatics fellowships have been well documented. However, less attention has been given to the posttraining experiences of graduates from informatics training programs. Here, we examine the career paths of subspecialty fellowship-trained pathology informaticians. METHODS: Alumni from four Pathology Informatics fellowship training programs were contacted for their voluntary participation in the study. We analyzed various components of training, and the subsequent career paths of Pathology Informatics fellowship alumni using data extracted from alumni provided curriculum vitae. RESULTS: Twenty-three out of twenty-seven alumni contacted contributed to the study. A majority had completed undergraduate study in science, technology, engineering, and math fields and combined track training in anatomic and clinical pathology. Approximately 30% (7/23) completed residency in a program with an in-house Pathology Informatics fellowship. Most completed additional fellowships (15/23) and many also completed advanced degrees (10/23). Common primary posttraining appointments included chief medical informatics officer (3/23), director of Pathology Informatics (10/23), informatics program director (2/23), and various roles in industry (3/23). Many alumni also provide clinical care in addition to their informatics roles (14/23). Pathology Informatics alumni serve on a variety of institutional committees, participate in national informatics organizations, contribute widely to scientific literature, and more than half (13/23) have obtained subspecialty certification in Clinical Informatics to date. CONCLUSIONS: Our analysis highlights several interesting phenomena related to the training and career trajectory of Pathology Informatics fellowship alumni. We note the long training track alumni complete in preparation for their careers. We believe flexible training pathways combining informatics and clinical training may help to alleviate the burden. We highlight the importance of in-house Pathology Informatics fellowships in promoting interest in informatics among residents. We also observe the many important leadership roles in academia, large community health systems, and industry available to early career alumni and believe this reflects a strong market for formally trained informaticians. We hope this analysis will be useful as we continue to develop the informatics fellowships to meet the future needs of our trainees and discipline.

Development of a Commercial Reference Laboratory Elective Rotation for Residents in Clinical Pathology.

OBJECTIVES: To develop a curriculum for a commercial reference laboratory clinical pathology training elective. METHODS: A 4-day elective at Quest Diagnostics was developed. The elective included 32 sessions composed of interactive didactic sessions and laboratory tours/demonstrations. Ten residents who attended the elective completed a written evaluation and scored each component of the curriculum. RESULTS: Written comments were very positive and demonstrated the goals of the elective were achieved. Laboratory tours and one-on-one sessions with the medical directors were especially well received. Most of the residents stated that the rotation gave them exposure to an area of laboratory medicine that they were not familiar with. CONCLUSIONS: The elective provided a resident training experience that was highly regarded and exposed residents to an area of laboratory medicine not encountered in most pathology training programs. Our curriculum could serve as a model for establishing a similar elective in other training programs.

Analysis of Daily Laboratory Orders at a Large Urban Academic Center: A Multifaceted Approach to Changing Test Ordering Patterns.

OBJECTIVES: We sought to address concerns regarding recurring inpatient laboratory test order practices (daily laboratory tests) through a multifaceted approach to changing ordering patterns. METHODS: We engaged in an interdepartmental collaboration to foster mindful test ordering through clinical policy creation, electronic clinical decision support, and continuous auditing and feedback. RESULTS: Annualized daily order volumes decreased from approximately 25,000 to 10,000 during a 33-month postintervention review. This represented a significant change from preintervention order volumes (95% confidence interval, 0.61-0.64; P < 10-16). Total inpatient test volumes were not affected. CONCLUSIONS: Durable changes to inpatient order practices can be achieved through a collaborative approach to utilization management that includes shared responsibility for establishing clinical guidelines and electronic decision support. Our experience suggests auditing and continued feedback are additional crucial components to changing ordering behavior. Curtailing daily orders alone may not be a sufficient strategy to reduce in-laboratory costs.