Electronic Health Records' Support for Primary Care Physicians' Situation Awareness: A Metanarrative Review.

OBJECTIVE: Situation awareness (SA) refers to people's perception and understanding of their dynamic environment. In primary care, reduced SA among physicians increases errors in clinical decision-making and, correspondingly, patients' risk of experiencing adverse outcomes. Our objective was to understand the extent to which electronic health records (EHRs) support primary care physicians (PCPs)' SA during clinical decision-making. METHOD: We conducted a metanarrative review of papers in selected academic databases, including CINAHL and MEDLINE. Eligible studies included original peer-reviewed research published between January 2012 and August 2020 on PCP-EHR interactions. We iteratively queried, screened, and summarized literature focused on EHRs supporting PCPs' clinical decision-making and care management for adults. Then, we mapped findings to an established SA framework to classify external factors (individual, task, and system) affecting PCPs' levels of SA (1-Perception, 2-Comprehension, and 3-Projection) and identified SA barriers. RESULTS: From 1504 articles identified, we included and synthesized 19 studies. Study designs were largely noninterventional. Studies described EHR workflow misalignments, usability issues, and communication challenges. EHR information, including lab results and care plans, was characterized as incomplete, untimely, or irrelevant. Unmet information needs made it difficult for PCPs to obtain even basic SA, Level 1 SA. Prevalent barriers to PCPs developing SA with EHRs were errant mental models, attentional tunneling, and data overload. CONCLUSION: Based on our review, EHRs do not support the development of higher levels of SA among PCPs. Review findings suggest SA-oriented design processes for health information technology could improve PCPs' SA, satisfaction, and decision-making.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps.

BACKGROUND: Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm. OBJECTIVE: As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff. DESIGN: We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety. SETTING: Twelve VA facilities across the USA. PARTICIPANTS: Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership. APPROACH: We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement. RESULTS: We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme. CONCLUSIONS: Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study.

BACKGROUND: Workload from electronic health record (EHR) inbox notifications leads to information overload and contributes to job dissatisfaction and physician burnout. Better understanding of physicians' inbox requirements and workflows could optimize inbox designs, enhance efficiency, and reduce safety risks from information overload. DESIGN: We conducted a mixed-methods study to identify strategies to enhance EHR inbox design and workflow. First, we performed a secondary analysis of national survey data of all Department of Veterans Affairs (VA) primary care practitioners (PCP) to identify major themes in responses to a free-text question soliciting suggestions to improve EHR inbox design and workflows. We then conducted expert interviews of clinicians at five health care systems (1 VA and 4 non-VA settings using 4 different EHRs) to understand existing optimal strategies to improve efficiency and situational awareness related to EHR inbox use. Themes from survey data were cross-validated with interview findings. RESULTS: We analyzed responses from 2104 PCPs who completed the free-text inbox question (of 5001 PCPs who responded to survey) and used an inductive approach to identify five themes: (1) Inbox notification content should be actionable for patient care and relevant to recipient clinician, (2) Inboxes should reduce risk of losing messages, (3) Inbox functionality should be optimized to improve efficiency of processing notifications, (4) Team support should be leveraged to help with EHR inbox notification burden, (5) Sufficient time should be provided to all clinicians to process EHR inbox notifications. We subsequently interviewed 15 VA and non-VA clinicians and identified 11 unique strategies, each corresponding directly with one of these five themes. CONCLUSION: Feedback from practicing end-user clinicians provides robust evidence to improve content and design of the EHR inbox and related clinical workflows and organizational policies. Several strategies we identified could improve clinicians' EHR efficiency and satisfaction as well as empower them to work with their local administrators, health IT personnel, and EHR developers to improve these systems.

Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer.

BACKGROUND & AIMS: Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC. METHODS: We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance. RESULTS: We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%. CONCLUSIONS: Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.

Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism.

BACKGROUND: Delays in following up abnormal test results are a common problem in outpatient settings. Surveillance systems that use trigger tools to identify delayed follow-up can help reduce missed opportunities in care. OBJECTIVE: To develop and test an electronic health record (EHR)-based trigger algorithm to identify instances of delayed follow-up of abnormal thyroid-stimulating hormone (TSH) results in patients being treated for hypothyroidism. DESIGN: We developed an algorithm using structured EHR data to identify patients with hypothyroidism who had delayed follow-up (>60 days) after an abnormal TSH. We then retrospectively applied the algorithm to a large EHR data warehouse within the Department of Veterans Affairs (VA), on patient records from two large VA networks for the period from January 1, 2011, to December 31, 2011. Identified records were reviewed to confirm the presence of delays in follow-up. KEY RESULTS: During the study period, 645,555 patients were seen in the outpatient setting within the two networks. Of 293,554 patients with at least one TSH test result, the trigger identified 1250 patients on treatment for hypothyroidism with elevated TSH. Of these patients, 271 were flagged as potentially having delayed follow-up of their test result. Chart reviews confirmed delays in 163 of the 271 flagged patients (PPV = 60.1%). CONCLUSIONS: An automated trigger algorithm applied to records in a large EHR data warehouse identified patients with hypothyroidism with potential delays in thyroid function test results follow-up. Future prospective application of the TSH trigger algorithm can be used by clinical teams as a surveillance and quality improvement technique to monitor and improve follow-up.

Development and Validation of Electronic Health Record-based Triggers to Detect Delays in Follow-up of Abnormal Lung Imaging Findings.

Purpose To develop an electronic health record (EHR)-based trigger algorithm to identify delays in follow-up of patients with imaging results that are suggestive of lung cancer and to validate this trigger on retrospective data. Materials and Methods The local institutional review board approved the study. A "trigger" algorithm was developed to automate the detection of delays in diagnostic evaluation of chest computed tomographic (CT) images and conventional radiographs that were electronically flagged by reviewing radiologists as being "suspicious for malignancy." The trigger algorithm was developed through literature review and expert input. It included patients who were alive and 40-70 years old, and it excluded instances in which appropriate timely follow-up (defined as occurring within 30 days) was detected (eg, pulmonary visit) or when follow-up was unnecessary (eg, in patients with a terminal illness). The algorithm was iteratively applied to a retrospective test cohort in an EHR data warehouse at a large Veterans Affairs facility, and manual record reviews were used to validate each individual criterion. The final algorithm aimed at detecting an absence of timely follow-up was retrospectively applied to an independent validation cohort to determine the positive predictive value (PPV). Trigger performance, time to follow-up, reasons for lack of follow-up, and cancer outcomes were analyzed and reported by using descriptive statistics. Results The trigger algorithm was retrospectively applied to the records of 89 168 patients seen between January 1, 2009, and December 31, 2009. Of 538 records with an imaging report that was flagged as suspicious for malignancy, 131 were identified by the trigger as being high risk for delayed diagnostic evaluation. Manual chart reviews confirmed a true absence of follow-up in 75 cases (trigger PPV of 57.3% for detecting evaluation delays), of which four received a diagnosis of primary lung cancer within the subsequent 2 years. Conclusion EHR-based triggers can be used to identify patients with suspicious imaging findings in whom follow-up diagnostic evaluation was delayed. (©) RSNA, 2015.

An electronic trigger to detect telemedicine-related diagnostic errors.

INTRODUCTION: The COVID-19 pandemic advanced the use of telehealth-facilitated care. However, little is known about how to measure safety of clinical diagnosis made through telehealth-facilitated primary care. METHODS: We used the seven-step Safer Dx Trigger Tool framework to develop an electronic trigger (e-trigger) tool to identify potential missed opportunities for more timely diagnosis during primary care telehealth visits at a large Department of Veterans Affairs facility. We then applied the e-trigger algorithm to electronic health record data related to primary care visits during a 1-year period (1 April 2020-31 March 2021). The algorithm identified patients with unexpected visits within 10 days of an index telemedicine visit and classified such records as e-trigger positive. We then validated the e-trigger's ability to detect missed opportunities in diagnosis using chart reviews based on a structured data collection instrument (the Revised Safer Dx instrument). RESULTS: We identified 128,761 telehealth visits (32,459 unique patients), of which 434 visits led to subsequent unplanned emergency department (ED), hospital, or primary care visits within 10 days of the index visit. Of these, 116 were excluded for clinical reasons (trauma, injury, or childbirth), leaving 318 visits (240 unique patients) needing further evaluation. From these, 100 records were randomly selected for review, of which four were falsely flagged due to invalid data (visits by non-providers or those incorrectly flagged as completed telehealth visits). Eleven patients had a missed opportunity in diagnosis, yielding a positive predictive value of 11%. DISCUSSION: Electronic triggers that identify missed opportunities for additional evaluation could help advance the understanding of safety of clinical diagnosis made in telehealth-enabled care. Better measurement can help determine which patients can safely be cared for via telemedicine versus traditional in-person visits.

Medical Home Implementation and Follow-Up of Cancer-Related Abnormal Test Results in the Veterans Health Administration.

Importance: Lack of timely follow-up of cancer-related abnormal test results can lead to delayed or missed diagnoses, adverse cancer outcomes, and substantial cost burden for patients. Care delivery models, such as the Veterans Affairs' (VA) Patient-Aligned Care Team (PACT), which aim to improve patient-centered care coordination, could potentially also improve timely follow-up of abnormal test results. PACT was implemented nationally in the VA between 2010 and 2012. Objective: To evaluate the long-term association between PACT implementation and timely follow-up of abnormal test results related to the diagnosis of 5 different cancers. Design, Setting, and Participants: This multiyear retrospective cohort study used 14 years of VA data (2006-2019), which were analyzed using panel data-based random-effects linear regressions. The setting included all VA clinics and facilities. The participants were adult patients who underwent diagnostic testing related to 5 different cancers and had abnormal test results. Data extraction and statistical analyses were performed from September 2021 to December 2023. Exposure: Calendar years denoting preperiods and postperiods of PACT implementation, and the PACT Implementation Progress Index Score denoting the extent of implementation in each VA clinic and facility. Main Outcome and Measure: Percentage of potentially missed timely follow-ups of abnormal test results. Results: This study analyzed 6 data sets representing 5 different types of cancers. During the initial years of PACT implementation (2010 to 2013), percentage of potentially missed timely follow-ups decreased between 3 to 7 percentage points for urinalysis suggestive of bladder cancer, 12 to 14 percentage points for mammograms suggestive of breast cancer, 19 to 22 percentage points for fecal tests suggestive of colorectal cancer, and 6 to 13 percentage points for iron deficiency anemia laboratory tests suggestive of colorectal cancer, with no statistically significant changes for α-fetoprotien tests and lung cancer imaging. However, these beneficial reductions were not sustained over time. Better PACT implementation scores were associated with a decrease in potentially missed timely follow-up percentages for urinalysis (0.3-percentage point reduction [95% CI, -0.6 to -0.1] with 1-point increase in the score), and laboratory tests suggestive of iron deficiency anemia (0.5-percentage point reduction [95% CI,-0.8 to -0.2] with 1-point increase in the score). Conclusions and Relevance: This cohort study found that implementation of PACT in the VA was associated with a potential short-term improvement in the quality of follow-up for certain test results. Additional multifaceted sustained interventions to reduce missed test results are required to prevent care delays.

Estimating the Economically Justifiable Price of Limited-Duration Treatment with Donanemab for Early Symptomatic Alzheimer's Disease in the United States.

INTRODUCTION: The goal of this economic model is to estimate an economically justifiable price (EJP) for using donanemab for the treatment of early symptomatic Alzheimer's disease (AD) in the United States based on clinical data from the phase 3 TRAILBLAZER-ALZ 2 trial (NCT04437511). METHODS: We adapted an AD Markov state-transition model developed by the Institute for Clinical and Economic Review to estimate the EJP for donanemab at different willingness-to-pay (WTP) thresholds from the health care system perspective and the societal perspective as co-base cases. RESULTS: Assuming a WTP threshold of $150,000 per quality-adjusted life-year (QALY) gained, the model estimates a 1-year (13-dose) EJP for donanemab of $80,538 from the health care system perspective and $91,126 from the societal perspective; at a WTP threshold of $100,000 per QALY gained, the model estimates a 1-year (13-dose) EJP for donanemab of $44,691 from the health care system perspective and $55,419 from the societal perspective. Mean total treatment costs per patient at the $150,000 per QALY gained EJP derived from the health care system perspective were estimated at $77,812 based on the average number of doses of donanemab patients received in the co-base case analysis. One-way sensitivity analysis (OWSA) indicated that treatment efficacy, disease severity at the time of treatment initiation, and duration of treatment effect were the main drivers of the potential EJP. CONCLUSIONS: Results from this modeling simulation informed by the TRAILBLAZER-ALZ 2 study support an EJP for limited-duration treatment with donanemab that exceeds per-dose list prices for currently available amyloid-targeting therapies, implying potentially lower lifetime costs and better value for money.

Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis.

PURPOSE: Missed and delayed cancer diagnoses are common, harmful, and often preventable. Automated measures of quality of cancer diagnosis are lacking but could identify gaps and guide interventions. We developed and implemented a digital quality measure (dQM) of cancer emergency presentation (EP) using electronic health record databases of two health systems and characterized the measure's association with missed opportunities for diagnosis (MODs) and mortality. METHODS: On the basis of literature and expert input, we defined EP as a new cancer diagnosis within 30 days after emergency department or inpatient visit. We identified EPs for lung cancer and colorectal cancer (CRC) in the Department of Veterans Affairs (VA) and Geisinger from 2016 to 2020. We validated measure accuracy and identified preceding MODs through standardized chart review of 100 records per cancer per health system. Using VA's longitudinal encounter and mortality data, we applied logistic regression to assess EP's association with 1-year mortality, adjusting for cancer stage and demographics. RESULTS: Among 38,565 and 2,914 patients with lung cancer and 14,674 and 1,649 patients with CRCs at VA and Geisinger, respectively, our dQM identified EPs in 20.9% and 9.4% of lung cancers, and 22.4% and 7.5% of CRCs. Chart reviews revealed high positive predictive values for EPs across sites and cancer types (72%-90%), and a substantial percent represented MODs (48.8%-84.9%). EP was associated with significantly higher odds of 1-year mortality for lung cancer and CRC (adjusted odds ratio, 1.78 and 1.83, respectively, 95% CI, 1.63 to 1.86 and 1.61 to 2.07). CONCLUSION: A dQM for cancer EP was strongly associated with both mortality and MODs. The findings suggest a promising automated approach to measuring quality of cancer diagnosis in US health systems.

Study protocol: maintaining preventive care during public health emergencies through effective coordination.

BACKGROUND: Screening lies at the heart of preventive care. However, COVID-19 dramatically disrupted routine screening efforts, resulting in excess mortality not directly attributable to COVID-19. Screening rates during COVID varied markedly by facility and clinical condition, suggesting susceptibilities in screening and referral process workflow. To better understand these susceptibilities and identify new practices to mitigate interrupted care, we propose a qualitative study comparing facilities that exhibited high, low, and highly variable performance (respectively) in screening rates before and during the pandemic. We will be guided by Weaver et al.'s multi-team systems (MTS) model of coordination, using cancer and mental health screening rates as exemplars. METHOD: Qualitative analysis of interviews and focus groups with primary care personnel, leadership, and patients at 10 VA medical centers. We will select sites based on rurality, COVID-19 caseload at the beginning of the pandemic, and performance on five outpatient clinical performance indicators of cancer and mental health screening. Sites will be categorized into one of five screening performance groups: high performers, low performers, improvers, plummeters, and highly variable. We will create process maps for each performance measure to create a workflow baseline and then interview primary care leadership to update the map at each site. We will clinician conduct focus groups to elicit themes regarding clinician coordination patterns (e.g., handoffs), strategies, and barriers/facilitators to screening during COVID. We will also conduct patient interviews to examine their screening experience during this period, for context. All interviews and focus groups will be audio-recorded, transcribed, and enhanced by field notes. We will analyze clinician transcripts and field notes using iterative, rapid analysis. Patient interviews will be analyzed using inductive/deductive content analysis. DISCUSSION: Our study represents a unique opportunity to inform the multi-team systems literature by identifying specific forms of information exchange, collective problem solving, and decision-making associated with higher and improved clinical performance. Specifically, our study aims to detect the specific points in the screening and referral process most susceptible to disruption and coordination processes that, if changed, will yield the highest value. Findings apply to future pandemics or any event with the potential to disrupt care.

Developing electronic clinical quality measures to assess the cancer diagnostic process.

OBJECTIVE: Measures of diagnostic performance in cancer are underdeveloped. Electronic clinical quality measures (eCQMs) to assess quality of cancer diagnosis could help quantify and improve diagnostic performance. MATERIALS AND METHODS: We developed 2 eCQMs to assess diagnostic evaluation of red-flag clinical findings for colorectal (CRC; based on abnormal stool-based cancer screening tests or labs suggestive of iron deficiency anemia) and lung (abnormal chest imaging) cancer. The 2 eCQMs quantified rates of red-flag follow-up in CRC and lung cancer using electronic health record data repositories at 2 large healthcare systems. Each measure used clinical data to identify abnormal results, evidence of appropriate follow-up, and exclusions that signified follow-up was unnecessary. Clinicians reviewed 100 positive and 20 negative randomly selected records for each eCQM at each site to validate accuracy and categorized missed opportunities related to system, provider, or patient factors. RESULTS: We implemented the CRC eCQM at both sites, while the lung cancer eCQM was only implemented at the VA due to lack of structured data indicating level of cancer suspicion on most chest imaging results at Geisinger. For the CRC eCQM, the rate of appropriate follow-up was 36.0% (26 746/74 314 patients) in the VA after removing clinical exclusions and 41.1% at Geisinger (1009/2461 patients; P < .001). Similarly, the rate of appropriate evaluation for lung cancer in the VA was 61.5% (25 166/40 924 patients). Reviewers most frequently attributed missed opportunities at both sites to provider factors (84 of 157). CONCLUSIONS: We implemented 2 eCQMs to evaluate the diagnostic process in cancer at 2 large health systems. Health care organizations can use these eCQMs to monitor diagnostic performance related to cancer.

The cost effectiveness of teriparatide as a first-line treatment for glucocorticoid-induced and postmenopausal osteoporosis patients in Sweden.

BACKGROUND: This paper presents the model and results to evaluate the use of teriparatide as a first-line treatment of severe postmenopausal osteoporosis (PMO) and glucocorticoid-induced osteoporosis (GIOP). The study's objective was to determine if teriparatide is cost effective against oral bisphosphonates for two large and high risk cohorts. METHODS: A computer simulation model was created to model treatment, osteoporosis related fractures, and the remaining life of PMO and GIOP patients. Natural mortality and additional mortality from osteoporosis related fractures were included in the model. Costs for treatment with both teriparatide and oral bisphosphonates were included. Drug efficacy was modeled as a reduction to the relative fracture risk for subsequent osteoporosis related fractures. Patient health utilities associated with age, gender, and osteoporosis related fractures were included in the model. Patient costs and utilities were summarized and incremental cost-effectiveness ratios (ICERs) for teriparatide versus oral bisphosphonates and teriparatide versus no treatment were estimated.For each of the PMO and GIOP populations, two cohorts differentiated by fracture history were simulated. The first contained patients with both a historical vertebral fracture and an incident vertebral fracture. The second contained patients with only an incident vertebral fracture. The PMO cohorts simulated had an initial Bone Mineral Density (BMD) T-Score of -3.0. The GIOP cohorts simulated had an initial BMD T-Score of -2.5. RESULTS: The ICERs for teriparatide versus bisphosphonate use for the one and two fracture PMO cohorts were €36,995 per QALY and €19,371 per QALY. The ICERs for teriparatide versus bisphosphonate use for the one and two fracture GIOP cohorts were €20,826 per QALY and €15,155 per QALY, respectively. CONCLUSIONS: The selection of teriparatide versus oral bisphosphonates as a first-line treatment for the high risk PMO and GIOP cohorts evaluated is justified at a cost per QALY threshold of €50,000.

Improving the effectiveness of electronic health record-based referral processes.

Electronic health records are increasingly being used to facilitate referral communication in the outpatient setting. However, despite support by technology, referral communication between primary care providers and specialists is often unsatisfactory and is unable to eliminate care delays. This may be in part due to lack of attention to how information and communication technology fits within the social environment of health care. Making electronic referral communication effective requires a multifaceted "socio-technical" approach. Using an 8-dimensional socio-technical model for health information technology as a framework, we describe ten recommendations that represent good clinical practices to design, develop, implement, improve, and monitor electronic referral communication in the outpatient setting. These recommendations were developed on the basis of our previous work, current literature, sound clinical practice, and a systems-based approach to understanding and implementing health information technology solutions. Recommendations are relevant to system designers, practicing clinicians, and other stakeholders considering use of electronic health records to support referral communication.

Translating electronic health record-based patient safety algorithms from research to clinical practice at multiple sites.

INTRODUCTION: Researchers are increasingly developing algorithms that impact patient care, but algorithms must also be implemented in practice to improve quality and safety. OBJECTIVE: We worked with clinical operations personnel at two US health systems to implement algorithms to proactively identify patients without timely follow-up of abnormal test results that warrant diagnostic evaluation for colorectal or lung cancer. We summarise the steps involved and lessons learned. METHODS: Twelve sites were involved across two health systems. Implementation involved extensive software documentation, frequent communication with sites and local validation of results. Additionally, we used automated edits of existing code to adapt it to sites' local contexts. RESULTS: All sites successfully implemented the algorithms. Automated edits saved sites significant work in direct code modification. Documentation and communication of changes further aided sites in implementation. CONCLUSION: Patient safety algorithms developed in research projects were implemented at multiple sites to monitor for missed diagnostic opportunities. Automated algorithm translation procedures can produce more consistent results across sites.

Cancer Evaluations During the COVID-19 Pandemic: An Observational Study Using National Veterans Affairs Data.

INTRODUCTION: Fewer cancer diagnoses have been made during the COVID-19 pandemic. Pandemic-related delays in cancer diagnosis could occur from limited access to care or patient evaluation delays (e.g., delayed testing after abnormal results). Follow-up of abnormal test results warranting evaluation for cancer was examined before and during the pandemic. METHODS: Electronic trigger algorithms were applied to the Department of Veterans Affairs electronic health record data to assess follow-up of abnormal test results before (March 10, 2019-March 7, 2020) and during (March 8, 2020-March 6, 2021) the pandemic. RESULTS: Electronic triggers were applied to 8,021,406 veterans' electronic health records to identify follow-up delays for abnormal results warranting evaluation for 5 cancers: bladder (urinalysis with high-grade hematuria), breast (abnormal mammograms), colorectal (positive fecal occult blood tests/fecal immunochemical tests or results consistent with iron deficiency anemia), liver (elevated alpha-fetoprotein), and lung (chest imaging suggestive of malignancy) cancers. Between prepandemic and pandemic periods, test quantities decreased by 12.6%-27.8%, and proportions of abnormal results lacking follow-up decreased for urinalyses (-0.8%), increased for fecal occult blood tests/fecal immunochemical test (+2.3%) and chest imaging (+1.8%), and remained constant for others. Follow-up times decreased for most tests; however, control charts suggested increased delays at 2 stages: early (pandemic beginning) for urinalyses, mammograms, fecal occult blood tests/fecal immunochemical test, iron deficiency anemia, and chest imaging and late (30-45 weeks into pandemic) for mammograms, fecal occult blood tests/fecal immunochemical test, and iron deficiency anemia. CONCLUSIONS: Although early pandemic delays in follow-up may have led to reduced cancer rates, the significant decrease in tests performed is likely a large driver of these reductions. Future emergency preparedness efforts should bolster essential follow-up and testing procedures to facilitate timely cancer diagnosis.

Assessing the Cost-effectiveness of a Hypothetical Disease-modifying Therapy With Limited Duration for the Treatment of Early Symptomatic Alzheimer Disease.

PURPOSE: Clinical trials have produced promising results for disease-modifying therapies (DMTs) for Alzheimer's disease (AD); however, the evidence on their potential cost-effectiveness is limited. This study assesses the cost-effectiveness of a hypothetical DMT with a limited treatment duration in AD. METHODS: We developed a Markov state-transition model to estimate the cost-effectiveness of a hypothetical DMT plus best supportive care (BSC) versus BSC alone among Americans living with mild cognitive impairment (MCI) due to AD or mild AD. AD states included MCI due to AD, mild AD, moderate AD, severe AD, and death. A hypothetical DMT was assumed to confer a 30% reduction in progression from MCI and mild AD. The base case annual drug acquisition cost was assumed to be $56,000. Other medical and indirect costs were obtained from published literature or list prices. Utilities for patients and caregivers were obtained from the published literature and varied by AD state and care setting (community care or long-term care). We considered 3 DMT treatment strategies: (1) treatment administered until patients reached severe AD (continuous strategy), (2) treatment administered for a maximum duration of 18 months or when patients reached severe AD (fixed-duration strategy), and (3) 40% of patients discontinuing treatment at 6 months because of amyloid plaque clearance and the remaining patients continuing treatment until 18 months or until they reached severe AD (test-and-discontinue strategy). Incremental cost-effectiveness ratios (ICERs) were calculated as the incremental cost per quality-adjusted life-year (QALY) gained. FINDINGS: From the health care sector perspective, continuous treatment with a hypothetical DMT versus BSC resulted in an ICER of $612,354 per QALY gained. The ICER decreased to $157,288 per QALY gained in the fixed-duration strategy, driven by large reductions in treatment costs. With 40% of patients discontinuing treatment at 6 months (test-and-discontinue strategy), the ICER was $125,631 per QALY gained. In sensitivity and scenario analyses, the ICER was the most sensitive to changes in treatment efficacy, treatment cost, and the initial population AD state distribution. From the modified societal perspective, ICERs were 6.3%, 20.4%, and 25.1% lower than those from the health care sector perspective for the continuous, fixed-duration, and test-and-discontinue strategies, respectively. IMPLICATIONS: Under a set of assumptions for annual treatment costs and the magnitude and duration of treatment efficacy, DMTs used for a limited duration may deliver value consistent with accepted US cost-effectiveness thresholds.

An evidence-based, structured, expert approach to selecting essential indicators of primary care quality.

BACKGROUND: The purpose of this article is to illustrate the application of an evidence-based, structured performance measurement methodology to identify, prioritize, and (when appropriate) generate new measures of health care quality, using primary care as a case example. Primary health care is central to the health care system and health of the American public; thus, ensuring high quality is essential. Due to its complexity, ensuring high-quality primary care requires measurement frameworks that can assess the quality of the infrastructure, workforce configurations, and processes available. This paper describes the use of the Productivity Measurement and Enhancement System (ProMES) to compile a targeted set of such measures, prioritized according to their contribution and value to primary care. METHODS: We adapted ProMES to select and rank existing primary care measures according to value to the primary care clinic. Nine subject matter experts (SMEs) consisting of clinicians, hospital leaders and national policymakers participated in facilitated expert elicitation sessions to identify objectives of performance, corresponding measures, and priority rankings. RESULTS: The SMEs identified three fundamental objectives: access, patient-health care team partnerships, and technical quality. The SMEs also selected sixteen performance indicators from the 44 pre-vetted, currently existing measures from three different data sources for primary care. One indicator, Team 2-Day Post Discharge Contact Ratio, was selected as an indicator of both team partnerships and technical quality. Indicators were prioritized according to value using the contingency functions developed by the SMEs. CONCLUSION: Our article provides an actionable guide to applying ProMES, which can be adapted to the needs of various industries, including measure selection and modification from existing data sources, and proposing new measures. Future work should address both logistical considerations (e.g., data capture, common data/programming language) and lingering measurement challenges, such as operationalizating measures to be meaningful and interpretable across health care settings.

Dashboards for visual display of patient safety data: a systematic review.

BACKGROUND: Methods to visualise patient safety data can support effective monitoring of safety events and discovery of trends. While quality dashboards are common, use and impact of dashboards to visualise patient safety event data remains poorly understood. OBJECTIVES: To understand development, use and direct or indirect impacts of patient safety dashboards. METHODS: We conducted a systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched PubMed, EMBASE and CINAHL for publications between 1 January 1950 and 30 August 2018 involving use of dashboards to display data related to safety targets defined by the Agency for Healthcare Research and Quality's Patient Safety Net. Two reviewers independently reviewed search results for inclusion in analysis and resolved disagreements by consensus. We collected data on development, use and impact via standardised data collection forms and analysed data using descriptive statistics. RESULTS: Literature search identified 4624 results which were narrowed to 33 publications after applying inclusion and exclusion criteria and consensus across reviewers. Publications included only time series and case study designs and were inpatient focused and emergency department focused. Information on direct impact of dashboards was limited, and only four studies included informatics or human factors principles in development or postimplementation evaluation. DISCUSSION: Use of patient-safety dashboards has grown over the past 15 years, but impact remains poorly understood. Dashboard design processes rarely use informatics or human factors principles to ensure that the available content and navigation assists task completion, communication or decision making. CONCLUSION: Design and usability evaluation of patient safety dashboards should incorporate informatics and human factors principles. Future assessments should also rigorously explore their potential to support patient safety monitoring including direct or indirect impact on patient safety.

Implementing Health Care Quality Measures in Electronic Health Records: A Conceptual Model.

BACKGROUND/OBJECTIVES: There is significant literature on the development and validation of quality measures, but comparably less on their implementation into learning health systems. Electronic Health Records (EHRs) have made vast amounts of data available for quality improvement purposes. In this paper we describe a conceptual model for EHR implementation of quality measures. DESIGN: The model involves five steps: (1) select a measure; (2) define measure criteria; (3) validate criteria and measurement process; (4) improve recording of measure-related activity; and (5) engage quality improvement processes. The model was used to develop and implement a quality measure in the Home-Based Medical Care (HBMC) setting. SETTING: Harris Health House Call Program (HHHC) provides primary medical and palliative care for homebound patients in Houston. PARTICIPANTS: Four-hundred twenty-four primary care patients followed in the HHHC. MEASUREMENT: Completion rate of the 9-item Patient Health Questionnaire (PHQ-9) within the Electronic Health Record of newly enrolled HHHC patients. RESULTS: Use of the conceptual model to guide implementation of a quality measure of depression screening in a HMBC practice was successful. Additional components of early leadership and clinician buy-in were required, as well as strong relationships with IT to ease implementation and limit disruptions in clinicians' work-flow. CONCLUSION: This conceptual model was feasible for guiding implementation of a quality measure for depression care of HBMC patients, and it can guide broader implementation of EHR-based quality measures in the future.