Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis.

PURPOSE: Missed and delayed cancer diagnoses are common, harmful, and often preventable. Automated measures of quality of cancer diagnosis are lacking but could identify gaps and guide interventions. We developed and implemented a digital quality measure (dQM) of cancer emergency presentation (EP) using electronic health record databases of two health systems and characterized the measure's association with missed opportunities for diagnosis (MODs) and mortality. METHODS: On the basis of literature and expert input, we defined EP as a new cancer diagnosis within 30 days after emergency department or inpatient visit. We identified EPs for lung cancer and colorectal cancer (CRC) in the Department of Veterans Affairs (VA) and Geisinger from 2016 to 2020. We validated measure accuracy and identified preceding MODs through standardized chart review of 100 records per cancer per health system. Using VA's longitudinal encounter and mortality data, we applied logistic regression to assess EP's association with 1-year mortality, adjusting for cancer stage and demographics. RESULTS: Among 38,565 and 2,914 patients with lung cancer and 14,674 and 1,649 patients with CRCs at VA and Geisinger, respectively, our dQM identified EPs in 20.9% and 9.4% of lung cancers, and 22.4% and 7.5% of CRCs. Chart reviews revealed high positive predictive values for EPs across sites and cancer types (72%-90%), and a substantial percent represented MODs (48.8%-84.9%). EP was associated with significantly higher odds of 1-year mortality for lung cancer and CRC (adjusted odds ratio, 1.78 and 1.83, respectively, 95% CI, 1.63 to 1.86 and 1.61 to 2.07). CONCLUSION: A dQM for cancer EP was strongly associated with both mortality and MODs. The findings suggest a promising automated approach to measuring quality of cancer diagnosis in US health systems.

Centralized Interactive Phenomics Resource: an integrated online phenomics knowledgebase for health data users.

OBJECTIVE: Development of clinical phenotypes from electronic health records (EHRs) can be resource intensive. Several phenotype libraries have been created to facilitate reuse of definitions. However, these platforms vary in target audience and utility. We describe the development of the Centralized Interactive Phenomics Resource (CIPHER) knowledgebase, a comprehensive public-facing phenotype library, which aims to facilitate clinical and health services research. MATERIALS AND METHODS: The platform was designed to collect and catalog EHR-based computable phenotype algorithms from any healthcare system, scale metadata management, facilitate phenotype discovery, and allow for integration of tools and user workflows. Phenomics experts were engaged in the development and testing of the site. RESULTS: The knowledgebase stores phenotype metadata using the CIPHER standard, and definitions are accessible through complex searching. Phenotypes are contributed to the knowledgebase via webform, allowing metadata validation. Data visualization tools linking to the knowledgebase enhance user interaction with content and accelerate phenotype development. DISCUSSION: The CIPHER knowledgebase was developed in the largest healthcare system in the United States and piloted with external partners. The design of the CIPHER website supports a variety of front-end tools and features to facilitate phenotype development and reuse. Health data users are encouraged to contribute their algorithms to the knowledgebase for wider dissemination to the research community, and to use the platform as a springboard for phenotyping. CONCLUSION: CIPHER is a public resource for all health data users available at https://phenomics.va.ornl.gov/ which facilitates phenotype reuse, development, and dissemination of phenotyping knowledge.

Developing electronic clinical quality measures to assess the cancer diagnostic process.

OBJECTIVE: Measures of diagnostic performance in cancer are underdeveloped. Electronic clinical quality measures (eCQMs) to assess quality of cancer diagnosis could help quantify and improve diagnostic performance. MATERIALS AND METHODS: We developed 2 eCQMs to assess diagnostic evaluation of red-flag clinical findings for colorectal (CRC; based on abnormal stool-based cancer screening tests or labs suggestive of iron deficiency anemia) and lung (abnormal chest imaging) cancer. The 2 eCQMs quantified rates of red-flag follow-up in CRC and lung cancer using electronic health record data repositories at 2 large healthcare systems. Each measure used clinical data to identify abnormal results, evidence of appropriate follow-up, and exclusions that signified follow-up was unnecessary. Clinicians reviewed 100 positive and 20 negative randomly selected records for each eCQM at each site to validate accuracy and categorized missed opportunities related to system, provider, or patient factors. RESULTS: We implemented the CRC eCQM at both sites, while the lung cancer eCQM was only implemented at the VA due to lack of structured data indicating level of cancer suspicion on most chest imaging results at Geisinger. For the CRC eCQM, the rate of appropriate follow-up was 36.0% (26 746/74 314 patients) in the VA after removing clinical exclusions and 41.1% at Geisinger (1009/2461 patients; P < .001). Similarly, the rate of appropriate evaluation for lung cancer in the VA was 61.5% (25 166/40 924 patients). Reviewers most frequently attributed missed opportunities at both sites to provider factors (84 of 157). CONCLUSIONS: We implemented 2 eCQMs to evaluate the diagnostic process in cancer at 2 large health systems. Health care organizations can use these eCQMs to monitor diagnostic performance related to cancer.

Framework of the Centralized Interactive Phenomics Resource (CIPHER) standard for electronic health data-based phenomics knowledgebase.

The development of phenotypes using electronic health records is a resource-intensive process. Therefore, the cataloging of phenotype algorithm metadata for reuse is critical to accelerate clinical research. The Department of Veterans Affairs (VA) has developed a standard for phenotype metadata collection which is currently used in the VA phenomics knowledgebase library, CIPHER (Centralized Interactive Phenomics Resource), to capture over 5000 phenotypes. The CIPHER standard improves upon existing phenotype library metadata collection by capturing the context of algorithm development, phenotyping method used, and approach to validation. While the standard was iteratively developed with VA phenomics experts, it is applicable to the capture of phenotypes across healthcare systems. We describe the framework of the CIPHER standard for phenotype metadata collection, the rationale for its development, and its current application to the largest healthcare system in the United States.

Cancer Evaluations During the COVID-19 Pandemic: An Observational Study Using National Veterans Affairs Data.

INTRODUCTION: Fewer cancer diagnoses have been made during the COVID-19 pandemic. Pandemic-related delays in cancer diagnosis could occur from limited access to care or patient evaluation delays (e.g., delayed testing after abnormal results). Follow-up of abnormal test results warranting evaluation for cancer was examined before and during the pandemic. METHODS: Electronic trigger algorithms were applied to the Department of Veterans Affairs electronic health record data to assess follow-up of abnormal test results before (March 10, 2019-March 7, 2020) and during (March 8, 2020-March 6, 2021) the pandemic. RESULTS: Electronic triggers were applied to 8,021,406 veterans' electronic health records to identify follow-up delays for abnormal results warranting evaluation for 5 cancers: bladder (urinalysis with high-grade hematuria), breast (abnormal mammograms), colorectal (positive fecal occult blood tests/fecal immunochemical tests or results consistent with iron deficiency anemia), liver (elevated alpha-fetoprotein), and lung (chest imaging suggestive of malignancy) cancers. Between prepandemic and pandemic periods, test quantities decreased by 12.6%-27.8%, and proportions of abnormal results lacking follow-up decreased for urinalyses (-0.8%), increased for fecal occult blood tests/fecal immunochemical test (+2.3%) and chest imaging (+1.8%), and remained constant for others. Follow-up times decreased for most tests; however, control charts suggested increased delays at 2 stages: early (pandemic beginning) for urinalyses, mammograms, fecal occult blood tests/fecal immunochemical test, iron deficiency anemia, and chest imaging and late (30-45 weeks into pandemic) for mammograms, fecal occult blood tests/fecal immunochemical test, and iron deficiency anemia. CONCLUSIONS: Although early pandemic delays in follow-up may have led to reduced cancer rates, the significant decrease in tests performed is likely a large driver of these reductions. Future emergency preparedness efforts should bolster essential follow-up and testing procedures to facilitate timely cancer diagnosis.

Translating electronic health record-based patient safety algorithms from research to clinical practice at multiple sites.

INTRODUCTION: Researchers are increasingly developing algorithms that impact patient care, but algorithms must also be implemented in practice to improve quality and safety. OBJECTIVE: We worked with clinical operations personnel at two US health systems to implement algorithms to proactively identify patients without timely follow-up of abnormal test results that warrant diagnostic evaluation for colorectal or lung cancer. We summarise the steps involved and lessons learned. METHODS: Twelve sites were involved across two health systems. Implementation involved extensive software documentation, frequent communication with sites and local validation of results. Additionally, we used automated edits of existing code to adapt it to sites' local contexts. RESULTS: All sites successfully implemented the algorithms. Automated edits saved sites significant work in direct code modification. Documentation and communication of changes further aided sites in implementation. CONCLUSION: Patient safety algorithms developed in research projects were implemented at multiple sites to monitor for missed diagnostic opportunities. Automated algorithm translation procedures can produce more consistent results across sites.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps.

BACKGROUND: Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm. OBJECTIVE: As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff. DESIGN: We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety. SETTING: Twelve VA facilities across the USA. PARTICIPANTS: Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership. APPROACH: We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement. RESULTS: We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme. CONCLUSIONS: Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

Lessons learned from an enterprise-wide clinical datathon.

In 2020, Baylor College of Medicine held a datathon to inform potential users of a new data warehouse, allow users to address clinical questions, identify warehouse capabilities and limitations, foster collaborations, and engage trainees. Senior faculty selected proposals based on feasibility and impact. Selectees worked with Information Technology for 2 months and presented findings. A survey of participants showed diverse levels of experience, high perceived value of the datathon, high rates of collaboration, and significant increases in knowledge. A datathon can promote familiarity with a new data warehouse, guide data warehouse improvement, and promote collaboration.

Acute kidney injury detection using refined and physiological-feature augmented urine output.

Acute kidney injury (AKI) is common in the intensive care unit, where it is associated with increased mortality. AKI is often defined using creatinine and urine output criteria. The creatinine-based definition is more reliable but less expedient, whereas the urine output based definition is rapid but less reliable. Our goal is to examine the urine output criterion and augment it with physiological features for better agreement with creatinine-based definitions of AKI. The objectives are threefold: (1) to characterize the baseline agreement of urine output and creatinine definitions of AKI; (2) to refine the urine output criteria to identify the thresholds that best agree with the creatinine-based definition; and (3) to build generalized estimating equation (GEE) and generalized linear mixed-effects (GLME) models with static and time-varying features to improve the accuracy of a near-real-time marker for AKI. We performed a retrospective observational study using data from two independent critical care databases, MIMIC-III and eICU, for critically ill patients who developed AKI in intensive care units. We found that the conventional urine output criterion (6 hr, 0.5 ml/kg/h) has specificity and sensitivity of 0.49 and 0.54 for MIMIC-III database; and specificity and sensitivity of 0.38 and 0.56 for eICU. Secondly, urine output thresholds of 12 hours and 0.6 ml/kg/h have specificity and sensitivity of 0.58 and 0.48 for MIMIC-III; and urine output thresholds of 10 hours and 0.6 ml/kg/h have specificity and sensitivity of 0.49 and 0.48 for eICU. Thirdly, the GEE model of four hours duration augmented with static and time-varying features can achieve a specificity and sensitivity of 0.66 and 0.61 for MIMIC-III; and specificity and sensitivity of 0.66 and 0.64 for eICU. The GLME model of four hours duration augmented with static and time-varying features can achieve a specificity and sensitivity of 0.71 and 0.55 for MIMIC-III; and specificity and sensitivity of 0.66 and 0.60 for eICU. The GEE model has greater performance than the GLME model, however, the GLME model is more reflective of the variables as fixed effects or random effects. The significant improvement in performance, relative to current definitions, when augmenting with patient features, suggest the need of incorporating these features when detecting disease onset and modeling at window-level rather than patient-level.

Effect of Pharmacogenetic Testing for Statin Myopathy Risk vs Usual Care on Blood Cholesterol: A Randomized Clinical Trial.

Importance: Nonadherence to statin guidelines is common. The solute carrier organic anion transporter family member 1B1 (SLCO1B1) genotype is associated with simvastatin myopathy risk and is proposed for clinical implementation. The unintended harms of using pharmacogenetic information to guide pharmacotherapy remain a concern for some stakeholders. Objective: To determine the impact of delivering SLCO1B1 pharmacogenetic results to physicians on the effectiveness of atherosclerotic cardiovascular disease (ASCVD) prevention (measured by low-density lipoprotein cholesterol [LDL-C] levels) and concordance with prescribing guidelines for statin safety and effectiveness. Design, Setting, and Participants: This randomized clinical trial was performed from December 2015 to July 2019 at 8 primary care practices in the Veterans Affairs Boston Healthcare System. Participants included statin-naive patients with elevated ASCVD risk. Data analysis was performed from October 2019 to September 2020. Interventions: SLCO1B1 genotyping and results reporting to primary care physicians at baseline (intervention group) vs after 1 year (control group). Main Outcomes and Measures: The primary outcome was the 1-year change in LDL-C level. The secondary outcomes were 1-year concordance with American College of Cardiology-American Heart Association and Clinical Pharmacogenetics Implementation Consortium (CPIC) guidelines for statin therapy and statin-associated muscle symptoms (SAMS). Results: Among 408 patients (mean [SD] age, 64.1 [7.8] years; 25 women [6.1%]), 193 were randomized to the intervention group and 215 were randomized to the control group. Overall, 120 participants (29%) had a SLCO1B1 genotype indicating increased simvastatin myopathy risk. Physicians offered statin therapy to 65 participants (33.7%) in the intervention group and 69 participants (32.1%) in the control group. Compared with patients whose physicians did not know their SLCO1B1 results at baseline, patients whose physicians received the results had noninferior reductions in LDL-C at 12 months (mean [SE] change in LDL-C, -1.1 [1.2] mg/dL in the intervention group and -2.2 [1.3] mg/dL in the control group; difference, -1.1 mg/dL; 90% CI, -4.1 to 1.8 mg/dL; P < .001 for noninferiority margin of 10 mg/dL). The proportion of patients with American College of Cardiology-American Heart Association guideline-concordant statin prescriptions in the intervention group was noninferior to that in the control group (12 patients [6.2%] vs 14 patients [6.5%]; difference, -0.003; 90% CI, -0.038 to 0.032; P < .001 for noninferiority margin of 15%). All patients in both groups were concordant with CPIC guidelines for safe statin prescribing. Physicians documented 2 and 3 cases of SAMS in the intervention and control groups, respectively, none of which was associated with a CPIC guideline-discordant prescription. Among patients with a decreased or poor SLCO1B1 transporter function genotype, simvastatin was prescribed to 1 patient in the control group but none in the intervention group. Conclusions and Relevance: Clinical testing and reporting of SLCO1B1 results for statin myopathy risk did not result in poorer ASCVD prevention in a routine primary care setting and may have been associated with physicians avoiding simvastatin prescriptions for patients at genetic risk for SAMS. Such an absence of harm should reassure stakeholders contemplating the clinical use of available pharmacogenetic results. Trial Registration: ClinicalTrials.gov Identifier: NCT02871934.

Natural Language Processing of Serum Protein Electrophoresis Reports in the Veterans Affairs Health Care System.

PURPOSE: Serum protein electrophoresis (SPEP) is a clinical tool used to screen for monoclonal gammopathy, thus it is a critical tool in the evaluation of patients with multiple myeloma. However, SPEP laboratory results are usually returned as short text reports, which are not amenable to simple computerized processing for large-scale studies. We applied natural language processing (NLP) to detect monoclonal gammopathy in SPEP laboratory results and compared its performance at multiple hospitals using both a rules-based manual system and a machine-learning algorithm. METHODS: We used the data from the VA Corporate Data Warehouse, which comprises data from 20 million unique individuals. SPEP reports were collected from July to December 2015 at 5 Veterans Affairs Medical Centers. Of these reports, we annotated the presence or absence of monoclonal gammopathy in 300 reports. We applied a machine learning-based NLP and a manual rules-based NLP to detect monoclonal gammopathy in SPEP reports at each of the hospitals, then applied the model from 1 hospital to each of the other hospitals. RESULTS: The learning system achieved an area under the receiver operating characteristic curve of 0.997, and the rules-based system achieved an accuracy of 0.99. When a model trained on 1 hospital's data was applied to a different hospital, however, accuracy varied greatly, and the learning-based models performed better than the rules-based model. CONCLUSION: Binary classification of short clinical texts such as SPEP reports may be a particularly attractive target on which to train highly accurate NLP systems.

Pragmatic Trials in Genomic Medicine: The Integrating Pharmacogenetics In Clinical Care (I-PICC) Study.

Pragmatic clinical trials (PCTs) have an established presence in clinical research and yet have only recently garnered attention within the landscape of genomic medicine. Using the PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) as a framework, this paper illustrates the application of PCT principles to The Integrating Pharmacogenetics In Clinical Care (I-PICC) Study, a trial of pharmacogenetic testing prior to statin initiation for cardiovascular disease prevention in primary care. The trial achieved high engagement with providers (85% enrolled of those approached) and enrolled a representative sample of participants for which statin therapy would be recommended. The I-PICC Study has a high level of pragmatism, which should enhance the generalizability of its findings. The PRECIS-2 may be useful in the design and evaluation of PCTs of genomic medicine interventions, contributing to the generation of evidence that can bridge the gap between genomics innovation and clinical adoption.

The Integrating Pharmacogenetics in Clinical Care (I-PICC) Study: Protocol for a point-of-care randomized controlled trial of statin pharmacogenetics in primary care.

BACKGROUND: The association between the SLCO1B1 rs4149056 variant and statin-associated muscle symptoms (SAMS) is well validated, but the clinical utility of its implementation in patient care is unknown. DESIGN: The Integrating Pharmacogenetics in Clinical Care (I-PICC) Study is a pseudo-cluster randomized controlled trial of SLCO1B1 genotyping among statin-naïve primary care and women's health patients across the Veteran Affairs Boston Healthcare System. Eligible patients of enrolled primary care providers are aged 40-75 and have elevated risk of cardiovascular disease by American College of Cardiology/American Heart Association (ACC/AHA) guidelines. Patients give consent by telephone in advance of an upcoming appointment, but they are enrolled only if and when their provider co-signs an order for SLCO1B1 testing, performed on a blood sample already collected in clinical care. Enrolled patients are randomly allocated to have their providers receive results through the electronic health record at baseline (PGx + arm) versus after 12 months (PGx- arm). The primary outcome is the change in low-density lipoprotein cholesterol (LDL-C) after one year. Secondary outcomes are concordance with Clinical Pharmacogenetics Implementation Consortium (CPIC) guidelines for simvastatin prescribing, concordance with ACC/AHA guidelines for statin use, and incidence of SAMS. With 408 patients, the study has >80% power to exclude a between-group LDL-C difference of 10 mg/dL (non-inferiority design) and to detect between-group differences of 15% in CPIC guideline concordance (superiority design). CONCLUSION: The outcomes of the I-PICC Study will inform the clinical utility of preemptive SLCO1B1 testing in the routine practice of medicine, including its proposed benefits and unforeseen risks.

Dynamic clinical data mining: search engine-based decision support.

The research world is undergoing a transformation into one in which data, on massive levels, is freely shared. In the clinical world, the capture of data on a consistent basis has only recently begun. We propose an operational vision for a digitally based care system that incorporates data-based clinical decision making. The system would aggregate individual patient electronic medical data in the course of care; query a universal, de-identified clinical database using modified search engine technology in real time; identify prior cases of sufficient similarity as to be instructive to the case at hand; and populate the individual patient's electronic medical record with pertinent decision support material such as suggested interventions and prognosis, based on prior outcomes. Every individual's course, including subsequent outcomes, would then further populate the population database to create a feedback loop to benefit the care of future patients.

Early detection of poor adherers to statins: applying individualized surveillance to pay for performance.

BACKGROUND: Medication nonadherence costs $300 billion annually in the US. Medicare Advantage plans have a financial incentive to increase medication adherence among members because the Centers for Medicare and Medicaid Services (CMS) now awards substantive bonus payments to such plans, based in part on population adherence to chronic medications. We sought to build an individualized surveillance model that detects early which beneficiaries will fall below the CMS adherence threshold. METHODS: This was a retrospective study of over 210,000 beneficiaries initiating statins, in a database of private insurance claims, from 2008-2011. A logistic regression model was constructed to use statin adherence from initiation to day 90 to predict beneficiaries who would not meet the CMS measure of proportion of days covered 0.8 or above, from day 91 to 365. The model controlled for 15 additional characteristics. In a sensitivity analysis, we varied the number of days of adherence data used for prediction. RESULTS: Lower adherence in the first 90 days was the strongest predictor of one-year nonadherence, with an odds ratio of 25.0 (95% confidence interval 23.7-26.5) for poor adherence at one year. The model had an area under the receiver operating characteristic curve of 0.80. Sensitivity analysis revealed that predictions of comparable accuracy could be made only 40 days after statin initiation. When members with 30-day supplies for their first statin fill had predictions made at 40 days, and members with 90-day supplies for their first fill had predictions made at 100 days, poor adherence could be predicted with 86% positive predictive value. CONCLUSIONS: To preserve their Medicare Star ratings, plan managers should identify or develop effective programs to improve adherence. An individualized surveillance approach can be used to target members who would most benefit, recognizing the tradeoff between improved model performance over time and the advantage of earlier detection.