A model to establish autoverification in the clinical laboratory.

OBJECTIVES: Autoverification is the process of evaluating and validating laboratory results using predefined computer-based algorithms without human interaction. By using autoverification, all reports are validated according to the standard evaluation criteria with predefined rules, and the number of reports per laboratory specialist is reduced. However, creating and validating these rules are the most demanding steps for setting up an autoverification system. In this study, we aimed to develop a model for helping users establish autoverification rules and evaluate their validity and performance. DESIGN & METHODS: The proposed model was established by analyzing white papers, previous study results, and national/international guidelines. An autoverification software (myODS) was developed to create rules according to the model and to evaluate the rules and autoverification rates. The simulation results that were produced by the software were used to demonstrate that the determined framework works as expected. Both autoverification rates and step-based evaluations were performed using actual patient results. Two algorithms defined according to delta check usage (Algorithm A and B) and three review limits were used for the evaluation. RESULTS: Six hundred seventeen rules were created according to the proposed model. 1,976 simulation results were created for validation. Our results showed that manual review limits are the most critical step in determining the autoverification rate, and delta check evaluation is especially important for evaluating inpatients. Algorithm B, which includes consecutive delta check evaluation, had higher AV rates. CONCLUSIONS: Systemic rule formation is a critical factor for successful AV. Our proposed model can help laboratories establish and evaluate autoverification systems. Rules created according to this model could be used as a starting point for different test groups.

Routine Health Information System (RHIS) improvements for strengthened health system management.

BACKGROUND: A well-functioning routine health information system (RHIS) can provide the information needed for health system management, for governance, accountability, planning, policy making, surveillance and quality improvement, but poor information support has been identified as a major obstacle for improving health system management. OBJECTIVES: To assess the effects of interventions to improve routine health information systems in terms of RHIS performance, and also, in terms of improved health system management performance, and improved patient and population health outcomes. SEARCH METHODS: We searched the Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, MEDLINE Ovid and Embase Ovid in May 2019. We searched Global Health, Ovid and PsycInfo in April 2016. In January 2020 we searched for grey literature in the Grey Literature Report and in OpenGrey, and for ongoing trials using the International Clinical Trials Registry Platform (ICTRP) and ClinicalTrials.gov. In October 2019 we also did a cited reference search using Web of Science, and a 'similar articles' search in PubMed. SELECTION CRITERIA: Randomised and non-randomised trials, controlled before-after studies and time-series studies comparing routine health information system interventions, with controls, in primary, hospital or community health care settings. Participants included clinical staff and management, district management and community health workers using routine information systems. DATA COLLECTION AND ANALYSIS: Two authors independently reviewed records to identify studies for inclusion, extracted data from the included studies and assessed the risk of bias. Interventions and outcomes were too varied across studies to allow for pooled risk analysis. We present a 'Summary of findings' table for each intervention comparisons broadly categorised into Technical and Organisational (or a combination), and report outcomes on data quality and service quality. We used the GRADE approach to assess the certainty of the evidence. MAIN RESULTS: We included six studies: four cluster randomised trials and two controlled before-after studies, from Africa and South America. Three studies evaluated technical interventions, one study evaluated an organisational intervention, and two studies evaluated a combination of technical and organisational interventions. Four studies reported on data quality and six studies reported on service quality. In terms of data quality, a web-based electronic TB laboratory information system probably reduces the length of time to reporting of TB test results, and probably reduces the overall rate of recording errors of TB test results, compared to a paper-based system (moderate certainty evidence). We are uncertain about the effect of the electronic laboratory information system on the recording rate of serious (misidentification) errors for TB test results compared to a paper-based system (very low certainty evidence). Misidentification errors are inaccuracies in transferring test results between an electronic register and patients' clinical charts. We are also uncertain about the effect of the intervention on service quality (timeliness of starting or changing a patient's TB treatment) (very low certainty evidence). A hand-held electronic device probably improves the length of time to report TB test results, and probably reduces the total frequency of recording errors in TB test results between the laboratory notebook and the electronic information record system, compared to a paper-based system (moderate-certainty evidence). We are, however, uncertain about the effect of the intervention on the frequency of serious (misidentification) errors in recording between the laboratory notebook and the electronic information record, compared to a paper-based system (very low certainty evidence). We are uncertain about the effect of a hospital electronic health information system on service quality (length of time outpatients spend at hospital, length of hospital stay, and hospital revenue collection), compared to a paper-based system (very low certainty evidence). High-intensity brief text messaging (SMS) may make little or no difference to data quality (in terms of completeness of documentation of pregnancy outcomes), compared to low-intensity brief text messaging (low-certainty evidence). We are uncertain about the effect of electronic drug stock notification (with either data management support or product transfer support) on service quality (in terms of transporting stock and stock levels), compared to paper-based stock notification (very low certainty evidence). We are uncertain about the effect of health information strengthening (where it is part of comprehensive service quality improvement intervention) on service quality (health worker motivation, receipt of training by health workers, health information index scores, quality of clinical observation of children and adults) (very low certainty evidence). AUTHORS' CONCLUSIONS: The review indicates mixed effects of mainly technical interventions to improve data quality, with gaps in evidence on interventions aimed at enhancing data-informed health system management. There is a gap in interventions studying information support beyond clinical management, such as for human resources, finances, drug supply and governance. We need to have a better understanding of the causal mechanisms by which information support may affect change in management decision-making, to inform robust intervention design and evaluation methods.

Designing and Validating Autoverification Rules for Hematology Analysis in Sysmex XN-9000 Hematology System.

BACKGROUND: Hematology analysis is a common test among patients in hospital. However, manual verification of hematology analysis is time consuming and tedious, with variation between inter-individual laboratory workers. This study was to establish and validate a set of autoverification rules for hematology analysis in the department of laboratory medicine, Zhongshan Hospital of Sun Yatsen University. METHODS: Hematology analysis was measured by a Sysmex XN-9000 hematology system in the Department of Laboratory Medicine, Zhongshan Hospital of Sun Yatsen University. SYSMEX Laboman EasyAccess 6.0 and the laboratory information system were used to construct the algorithm and design the autoverification rules of hematology analysis according to Clinical and Laboratory Standards Institute document Auto 10A and 41 rules of Hematology Review Criteria. The laboratory turnaround time (TAT), autoverification pass rates, false positive, false negative, and the average error rate were verified after implementing autoverification rules. RESULTS: Approximate 1,300 specimens were collected daily and transferred to our laboratory for hematology analysis; that is necessary to build a database and to design autoverification rules. The average autoverification passing rate was 81%; the false positive rate was 13.6%; the false negative rate and the average error rate was nearly zero, indicating that incorrect reports were almost eliminated. Moreover, since implementing autoverification, the TAT was reduced by 27.0% in in-patient reports, by 21.9% in out-patient reports, and by 39.0% in emergency reports, which enhanced the productivity in our laboratory. CONCLUSIONS: Our laboratory accelerated verification and decreased TAT and the odds of human review errors in the released results since implementing the autoverification. Thus, we can save more time and concentrate on verifying the abnormal results and processing emergency tests.

Design of a state of the art reporting system and process improvement for reporting of high complexity single antigen bead data for transplant patients to the electronic medical record.

The UCLA Immunogenetics Center is an Immunogenetics and Histocompatibility laboratory that performs testing for multiple transplant programmes within and outside of UCLA. The single antigen bead (SAB) test is a high complexity luminex bead test used to assess pretransplant and post-transplant patients for the presence of pathogenic human leucocyte antigen donor-specific antibody associated with allograft rejection. Efficient reporting of the SAB test has been difficult as data analysis and reports are generated in the laboratory information system (LIS) and uploaded to the electronic medical record (EMR) as PDFs. To solve this, we recently developed a state of the art reporting workflow allowing discrete reporting of SAB data (antibody specificity, mean fluorescent intensity and interpretative comments) from the LIS HistoTrac to UCLA Health System's EMR EPIC:CareConnect. However, a proportion of tests did not report to the EMR appropriately. Baseline system performance data evaluated over a 10-week period showed that ~4.5/100 tests resulted in EPIC as 'preliminary result' or 'in process' instead of 'final result' with only common cause variation. Quality improvement methods were employed to improve the process with the SMART Aim of reporting 100% of tests as 'final result'. Pareto analysis identified two errors accounting for 79% of common system-level failures-status errors and interface errors. We hypothesised that addressing the status error would reduce or eliminate the interface errors. We used the Model For Improvement to test a reprogramming intervention. Status and interface errors were completely resolved through the process improvement. Continuous monitoring revealed a system-level shift with only ~1.9/100 tests resulting inappropriately. Through the audit process, the remaining common system-level failures were identified and resolved. Therefore, 100% of tests result to EPIC as 'final result'. The study demonstrates that high complexity SAB bead data can be efficiently reported EPIC:CareConnect from HistoTrac as discrete data.

Communication of Critical Laboratory Values: Optimization of the Process through Secure Messaging.

BACKGROUND: Timely communication of critical laboratory results is important yet cumbersome. OBJECTIVE: To assess the impact of a new technology on the process of reporting critical laboratory results at our 480-bed, adult/children, tertiary-care, medical school-affiliated health center in the southeastern region of the United States. METHODS: We changed the process of reporting critical values by telephone only to reporting via telephone and a secure messaging app. Physician order entry, an online on-call roster for availability, and support from the C-suite (executive branch of the organization) were instrumental in implementation. RESULTS: Consistently, before our process changes, more than 95% of the critical laboratory results were reported in less than 30 minutes. Use of the app reduced the time taken for reporting results. The need to involve pathology residents and attending physicians in reporting has been eliminated by this process. DISCUSSION: Secure messaging has facilitated the reporting of critical laboratory values, making it more efficient and providing a reliable record of the process. This process meets or exceeds the standards of the accrediting agencies. The method is suitable for activating rapid-response teams in case of hypercritical values.

The Effect of Information Technology on the Information Exchange between Laboratories and Ambulatory Care Centers: A Systematic Review.

Laboratory services form an integral part of medical care in the decision-making of physicians, including those working at ambulatory care centers. Information exchange is essential between ambulatory care centers and laboratories. Inevitable errors have always existed in the exchange of such information on paper, which can be to some extent avoided by developing appropriate computer-based interfaces. Therefore, this review aimed to examine studies conducted to determine the effect of electronic communication between ambulatory care centers and laboratories. This systematic review was conducted on the basis of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. Studies were searched in the PubMed, Embase, Cochrane, and Web of Science, and those written in English and published between 2000 and February 2019 with full texts available were selected. From a total of 3898 papers retrieved from the studied databases, 24 papers were eligible for entering this study after removing similar and nonrelated studies. Electronic exchanges between ambulatory care centers and laboratories can have numerous benefits in terms of financial, organizational, and quality. This evidence for the value of electronic communications is an important factor contributing to its local investment and adoption.

Autoverification of test results in the core clinical laboratory.

Verification of laboratory test results represents the last opportunity to identify errors before they become part of the electronic medical record. Manual verification of test results places significant reliance on the experience and attentiveness of individual observers to identify errors and is vulnerable to errors through omission and neglect. Peer-reviewed publications have documented gains in process efficiency and quality improvement by use of middleware or laboratory information systems to autoverify test results based on pre-defined acceptability criteria. This review evaluates the acceptability of autoverification (AV) as a safe and reliable alternative to total manual review of laboratory test results. AV schemes developed in accordance with international guidelines and standards are applied throughout the laboratory. Careful design of AV systems involves using multidisciplinary teams to develop test-specific decision algorithms, to assist with programming, to verify programming, and validate programmed algorithms prior to use in evaluation of patient test result profiles. Development of test specific decision algorithms makes use of criteria based on instrument messages and flags, quality control status, result limit checks, delta checks, critical values, consistency checks, and patient-related clinical information. Monitoring of the performance of AV parameters, and regular audits of the AV system integrity is recommended in both the literature and guidelines. The potential for gains to process efficiency, error detection and patient safety, through adoption of AV as part of a laboratories quality assurance tool-case, is well supported in published literature.

Design and evaluation of a LIS-based autoverification system for coagulation assays in a core clinical laboratory.

BACKGROUND: The autoverification system for coagulation consists of a series of rules that allow normal data to be released without manual verification. With new advances in medical informatics, the laboratory information system (LIS) has growing potential for the autoverification, allowing rapid and accurate verification of clinical laboratory tests. The purpose of the study is to develop and evaluate a LIS-based autoverification system for validation and efficiency. METHODS: Autoverification decision rules, including quality control, analytical error flag, critical value, limited range check, delta check and logical check, as well as patient's historical information, were integrated into the LIS. Autoverification limited range was constructed based on 5 and 95% percentiles. The four most commonly used coagulation assays, prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT), and fibrinogen (FBG), were followed by the autoverification protocols. The validation was assessed by the autoverification passing rate, the true-positive cases, the true-negative cases, the false-positive cases, the false-negative cases, the sensitivity and the specificity; the efficiency was evaluated in the turnaround time (TAT). RESULTS: A total of 157,079 historical test results of coagulation profiles from January 2016 to December 2016 were collected to determine the distribution intervals. The autoverification passing rate was 77.11% (29,165/37,821) based on historical patient data. In the initial test of the autoverification version in June 2017, the overall autoverification passing rate for the whole sample was 78.75% (11,257/14,295), with 892 true-positive cases, 11,257 true-negative cases, 2146 false-positive cases, no false-negative cases, sensitivity of 100% and specificity of 83.99%. After formal implementation of the autoverification system for 6 months, 83,699 samples were assessed. The average overall autoverification passing rate for the whole sample was 78.86% and the 95% confidence interval (CI) of the passing rate was [78.25, 79.59%]. TAT was reduced from 126 min to 101 min, which was statistically significant (P < 0.001, Mann-Whitney U test). CONCLUSIONS: The autoverification system for coagulation assays based on LIS can halt the samples with abnormal values for manual verification, guarantee medical safety, minimize the requirements for manual work, shorten TAT and raise working efficiency.

Semantic data interoperability, digital medicine, and e-health in infectious disease management: a review.

Disease management requires the use of mixed languages when discussing etiology, diagnosis, treatment, and follow-up. All phases require data management, and, in the optimal case, such data are interdisciplinary and uniform and clear to all those involved. Such semantic data interoperability is one of the technical building blocks that support emerging digital medicine, e-health, and P4-medicine (predictive, preventive, personalized, and participatory). In a world where infectious diseases are on a trend to become hard-to-treat threats due to antimicrobial resistance, semantic data interoperability is part of the toolbox to fight more efficiently against those threats. In this review, we will introduce semantic data interoperability, summarize its added value, and analyze the technical foundation supporting the standardized healthcare system interoperability that will allow moving forward to e-health. We will also review current usage of those foundational standards and advocate for their uptake by all infectious disease-related actors.

Establishing and Evaluating Autoverification Rules with Intelligent Guidelines for Arterial Blood Gas Analysis in a Clinical Laboratory.

Arterial blood gas (ABG) analysis is important for acutely ill patients and should be performed by qualified laboratorians. The existing manual verifications are tedious, time-consuming, and prone to send wrong reports. Autoverification uses computer-based rules to verify clinical laboratory test results without manual review. To date, no data are available on the use of autoverification for ABG analysis. All autoverification rules were established according to AUTO10-A. Additionally, the rules were established using retrospective patient data, and then validated by actual clinical samples in a "live" environment before go-live. The average autoverification passing rate was 75.5%. The turnaround time (TAT) was reduced by 33.3% (27 min vs 18 min). Moreover, the error rate fell to 0.05% after implementation. Statistical analysis resulted in a kappa statistic of 0.92 ( p < 0.01), indicating close agreement between autoverification and senior technician verification, and the chi-square value was 22.4 ( p < 0.01), indicating that the autoverification error rate was lower than the manual verification error rate. Results showed that implementing autoverification rules with intelligent guidelines for ABG analysis of patients with critical illnesses could decrease the number of samples requiring manual verification, reduce TAT, and eliminate errors, allowing laboratorians to concentrate more time on abnormal samples, patient care, and collaboration with physicians.

Implementation of the Autovalidation Algorithm for Clinical Chemistry Testing in the Laboratory Information System.

OBJECTIVE: Autovalidation algorithm should be properly designed with clearly defined criteria and any data that do not meet the criteria, must be reviewed and manually validated. The aim was to define the rules for autovalidation in our laboratory information system (LIS), and validate the algorithm prior to its implementation in routine laboratory work. METHODS: Autovalidation was implemented for all routine serum biochemistry tests. The algorithm included analytical measurement ranges (AMR), delta check, critical values, serum indices and all preanalytical and analytical flags from the analyzer. RESULTS: In the validation process 9805 samples were included, and 78.3% (7677) of all samples were autovalidated. The highest percentage of non-validated samples (54.9%) refers to those with at least one result outside the method linearity ranges (AMR criteria) while critical values were observed to be the least frequent criterion for stopping autovalidation (1.8%). Also, 38 samples were manually validated as they failed to meet the autovalidation criteria. CONCLUSION: Implementation of algorithm for autovalidation in our institution resulted in the redesign of the existing LIS. This model of the autovalidation algorithm significantly decreased the number of manually validated test results and can be used as a model for introducing autovalidation in other laboratory settings.

LabRS: A Rosetta stone for retrospective standardization of clinical laboratory test results.

Objective: Clinical laboratories in the United States do not have an explicit result standard to report the 7 billion laboratory tests results they produce each year. The absence of standardized test results creates inefficiencies and ambiguities for secondary data users. We developed and tested a tool to standardize the results of laboratory tests in a large, multicenter clinical data warehouse. Methods: Laboratory records, each of which consisted of a laboratory result and a test identifier, from 27 diverse facilities were captured from 2000 through 2015. Each record underwent a standardization process to convert the original result into a format amenable to secondary data analysis. The standardization process included the correction of typos, normalization of categorical results, separation of inequalities from numbers, and conversion of numbers represented by words (eg, "million") to numerals. Quality control included expert review. Results: We obtained 1.266 × 109 laboratory records and standardized 1.252 × 109 records (98.9%). Of the unique unstandardized records (78.887 × 103), most appeared <5 times (96%, eg, typos), did not have a test identifier (47%), or belonged to an esoteric test with <100 results (2%). Overall, these 3 reasons accounted for nearly all unstandardized results (98%). Conclusion: Current results suggest that the tool is both scalable and generalizable among diverse clinical laboratories. Based on observed trends, the tool will require ongoing maintenance to stay current with new tests and result formats. Future work to develop and implement an explicit standard for test results would reduce the need to retrospectively standardize test results.

Indirect method for validating transference of reference intervals.

BACKGROUND: Transference of reference intervals (RIs) from multicentre studies are often verified by use of a small number of samples from reference individuals or by the use of one serum sample (Serum X for NORIP RI). Despite recommended and appropriate methods, both have inconveniencies and drawbacks. Several attempts have been made to develop an indirect method, which uses historical data from the laboratory. These methods are retrospective relying on older test results. A near prospective method would be preferable for the laboratories introducing new methods or changing analytical platforms. METHODS: We performed a data mining experiment using results from our laboratory information system covering patients from a large geographic area. Request patterns for patients with assumed healthy characteristics were identified and used to extract laboratory results for calculation of new RI by an indirect method. Calculated RI and confidence intervals (CIs) were compared to transferred NORIP RI verified by NFKK Reference Serum X. RESULTS: We found that our indirect method and NFKK Reference Serum X in general produced similar results when verifying transference of RI. The method produces results for all stratifications. Only single stratifications and one analyte showed unexplained incongruences to the NORIP RI. CONCLUSIONS: Our results suggest using request patterns as a surrogate measure for good health status. This allows for a data mining method for validation of RI or validating their transference, which is likely to be applicable in countries with similar healthcare and laboratory information system.

Estado de la certificación/acreditación de los laboratorios clínicos españoles: estudio inicial / State of certification/accreditation in Spanish clinical laboratorios: an initial study

No disponible

Internal quality control in an academic cytopathology laboratory for the introduction of a new reporting system for endometrial cytology.

BACKGROUND: To evaluate reproducibility of a reporting system for endometrial cytology. METHODS: Cytologic slides from 49 patients, prepared via liquid based cytology, were blindly examined by five cytopathologists of various experience levels, applying a recently introduced reporting system as previously reported. The agreement among cytopathologists was evaluated via Kappa (κ) statistics and the Kendall's Coefficient of Variation (W); cytologic results were compared with the relevant histologic report. RESULTS: Substantial agreement among all five raters was found in the benign, ACE-L and malignant categories, fair agreement in inadequate and ACE-H categories, whereas only slight agreement in ACE-U. For the three more experienced cytopathologists, an almost perfect agreement was found in inadequate, benign, and ACE-L categories, substantial agreement in ACE-H and malignant categories and fair agreement in ACE-U category. Overall agreement for all five cytopathologists and for all categories was moderate, whereas it was very high for the three senior raters. Using the Kendall's test, both five cytopathologists (W = 0.81) and the three senior ones (W = 0.93) had very high agreement. Sensitivity: 83.33-92.59%, specificity: 83.33-94.74%, ROC area: 71.72-90.3%. CONCLUSION: Application of appropriate statistical tests shows that integration of a new reporting cytologic system is effective with an overall accuracy around 90%. Both statistical tests applied disclosed lower agreement rates among both all five raters and the three most experienced ones in the intermediate categories constituting the gray zone, thus delineating the need for better training of cytopathologists to correctly identify diagnostic criteria for classification of a given case into these categories.

Impact of reference change value (RCV) based autoverification on turnaround time and physician satisfaction.

BACKGROUND: For a quicker delivery of laboratory test results to the hospital emergency department (ED), we implemented an autoverification system based on the reference change value (RCV). The aim of this study was to assess how the RCV based autoverification reflected on turnaround time (TAT) and on physician satisfaction. MATERIALS AND METHODS: The laboratory information system (LIS) was programmed to autoverify the results as long as they were within the range settled by RCV, so that the autoverified results were reported to the physician as soon as the tests were carried out, without any further intervention. We analyzed the same three-month periods' TAT and verification time (VFT) from the years prior to and following the implementation of RCV autoverification. The change in physicians' satisfaction levels was assessed using the hospital's Annual Physician Satisfaction Survey (APSS). Over sixty percent of physicians completed the questionnaire, and the amount of daily ED test requests (nearly three hundred) did not vary throughout the duration of this study. RESULTS: Mann-Whitney U test showed that the VFT was significantly reduced in all the test but troponin I. There were substantial reductions in TAT medians (haemogram, 75%; fibrinogen, 41%; prothrombin time, 40%; sodium, 27%). The percentage of physicians satisfied with the haematological and biochemical tests´ TAT increased from 84% to 93% and from 86% to 91% respectively. CONCLUSIONS: Our results reveal that VFT and TAT were severely reduced in most emergency tests, greatly improving physicians' satisfaction with TAT.

Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar.

PURPOSE: Data sharing through ClinVar offers a unique opportunity to identify interpretation differences between laboratories. As part of a ClinGen initiative, four clinical laboratories (Ambry, GeneDx, Partners Healthcare Laboratory for Molecular Medicine, and University of Chicago Genetic Services Laboratory) collaborated to identify the basis of interpretation differences and to investigate if data sharing and reassessment resolve interpretation differences by analyzing a subset of variants. METHODS: ClinVar variants with submissions from at least two of the four participating laboratories were compared. For a subset of identified differences, laboratories documented the basis for discordance, shared internal data, independently reassessed with the American College of Medical Genetics and Genomics-Association for Molecular Pathology (ACMG-AMP) guidelines, and then compared interpretations. RESULTS: At least two of the participating laboratories interpreted 6,169 variants in ClinVar, of which 88.3% were initially concordant. Laboratories reassessed 242/724 initially discordant variants, of which 87.2% (211) were resolved by reassessment with current criteria and/or internal data sharing; 12.8% (31) of reassessed variants remained discordant owing to differences in the application of the ACMG-AMP guidelines. CONCLUSION: Participating laboratories increased their overall concordance from 88.3 to 91.7%, indicating that sharing variant interpretations in ClinVar-thereby allowing identification of differences and motivation to resolve those differences-is critical to moving toward more consistent variant interpretations.Genet Med advance online publication 09 March 2017.