Managing Interruptions to Improve Diagnostic Decision-Making: Strategies and Recommended Research Agenda.

Interruptions are an inevitable occurrence in health care. Interruptions in diagnostic decision-making are no exception and can have negative consequences on both the decision-making process and well-being of the decision-maker. This may result in inaccurate or delayed diagnoses. To date, research specific to interruptions on diagnostic decision-making has been limited, but strategies to help manage the negative impacts of interruptions need to be developed and implemented. In this perspective, we first present a modified model of interruptions to visualize the interruption process and illustrate where potential interventions can be implemented. We then consider several empirically tested strategies from the fields of health care and cognitive psychology that can lay the groundwork for additional research to mitigate effects of interruptions during diagnostic decision-making. We highlight strategies to minimize the negative impacts of interruptions as well as strategies to prevent interruptions altogether. Additionally, we build upon these strategies to propose specific research priorities within the field of diagnostic safety. Identifying effective interventions to help clinicians better manage interruptions has the potential to minimize diagnostic errors and improve patient outcomes.

Electronic Health Records' Support for Primary Care Physicians' Situation Awareness: A Metanarrative Review.

OBJECTIVE: Situation awareness (SA) refers to people's perception and understanding of their dynamic environment. In primary care, reduced SA among physicians increases errors in clinical decision-making and, correspondingly, patients' risk of experiencing adverse outcomes. Our objective was to understand the extent to which electronic health records (EHRs) support primary care physicians (PCPs)' SA during clinical decision-making. METHOD: We conducted a metanarrative review of papers in selected academic databases, including CINAHL and MEDLINE. Eligible studies included original peer-reviewed research published between January 2012 and August 2020 on PCP-EHR interactions. We iteratively queried, screened, and summarized literature focused on EHRs supporting PCPs' clinical decision-making and care management for adults. Then, we mapped findings to an established SA framework to classify external factors (individual, task, and system) affecting PCPs' levels of SA (1-Perception, 2-Comprehension, and 3-Projection) and identified SA barriers. RESULTS: From 1504 articles identified, we included and synthesized 19 studies. Study designs were largely noninterventional. Studies described EHR workflow misalignments, usability issues, and communication challenges. EHR information, including lab results and care plans, was characterized as incomplete, untimely, or irrelevant. Unmet information needs made it difficult for PCPs to obtain even basic SA, Level 1 SA. Prevalent barriers to PCPs developing SA with EHRs were errant mental models, attentional tunneling, and data overload. CONCLUSION: Based on our review, EHRs do not support the development of higher levels of SA among PCPs. Review findings suggest SA-oriented design processes for health information technology could improve PCPs' SA, satisfaction, and decision-making.

Advancing Diagnostic Safety Research: Results of a Systematic Research Priority Setting Exercise.

BACKGROUND: Diagnostic errors are a major source of preventable harm but the science of reducing them remains underdeveloped. OBJECTIVE: To identify and prioritize research questions to advance the field of diagnostic safety in the next 5 years. PARTICIPANTS: Ninety-seven researchers and 42 stakeholders were involved in the identification of the research priorities. DESIGN: We used systematic prioritization methods based on the Child Health and Nutrition Research Initiative (CHNRI) methodology. We first invited a large international group of expert researchers in various disciplines to submit research questions while considering five prioritization criteria: (1) usefulness, (2) answerability, (3) effectiveness, (4) potential for translation, and (5) maximal potential for effect on diagnostic safety. After consolidation, these questions were prioritized at an in-person expert meeting in April 2019. Top-ranked questions were subsequently reprioritized through scoring on the five prioritization criteria using an online questionnaire. We also invited non-research stakeholders to assign weights to the five criteria and then used these weights to adjust the final prioritization score for each question. KEY RESULTS: Of the 207 invited researchers, 97 researchers responded and 78 submitted 333 research questions which were then consolidated. Expert meeting participants (n = 21) discussed questions in different breakout sessions and prioritized 50, which were subsequently reduced to the top 20 using the online questionnaire. The top 20 questions addressed mostly system factors (e.g., implementation and evaluation of information technologies), teamwork factors (e.g., role of nurses and other health professionals in the diagnostic process), and strategies to engage patients in the diagnostic process. CONCLUSIONS: Top research priorities for advancing diagnostic safety in the short-term include strengthening systems and teams and engaging patients to support diagnosis. High-priority areas identified using these systematic methods can inform an actionable research agenda for reducing preventable diagnostic harm.

Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study.

BACKGROUND: Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful. OBJECTIVE: The study aimed to examine patients' experiences using an artificial intelligence (AI)-assisted online symptom checker. METHODS: An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected. RESULTS: A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool's results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool's results (20/34, 59% vs 22/69, 32%; P=.04). CONCLUSIONS: Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.

Pediatric clinician perspectives on communicating diagnostic uncertainty.

OBJECTIVE: Diagnosis often evolves over time, involves uncertainty, and is vulnerable to errors. We examined pediatric clinicians' perspectives on communicating diagnostic uncertainty to patients' parents and how this occurs. DESIGN: We conducted semi-structured interviews, which were audiotaped, transcribed, and analyzed using content analysis. Two researchers independently coded transcripts and then discussed discrepancies to reach consensus. SETTING: A purposive sample of pediatric clinicians at two large academic medical institutions in Texas. PARTICIPANTS: Twenty pediatric clinicians participated: 18 physicians, 2 nurse practitioners; 7 males, 13 females; 7 inpatient, 11 outpatient, and 2 practicing in mixed settings; with 0-16 years' experience post-residency. INTERVENTION(S): None. MAIN OUTCOME MEASURE(S): Pediatric clinician perspectives on communication of diagnostic uncertainty. RESULTS: Pediatric clinicians commonly experienced diagnostic uncertainty and most were comfortable seeking help and discussing with colleagues. However, when communicating uncertainty to parents, clinicians used multiple considerations to adjust the degree to which they communicated. Considerations included parent characteristics (education, socioeconomic status, emotional response, and culture) and strength of parent-clinician relationships. Communication content included setting expectations, explaining the diagnostic process, discussing most relevant differentials, and providing reassurance. Responses to certain parent characteristics, however, were variable. For example, some clinicians were more open to discussing diagnostic uncertainty with more educated parents- others were less. CONCLUSIONS: While pediatric clinicians are comfortable discussing diagnostic uncertainty with colleagues, how they communicate uncertainty to parents appears variable. Parent characteristics and parent-clinician relationships affect extent of communication and content discussed. Development and implementation of optimal strategies for managing and communicating diagnostic uncertainty can improve the diagnostic process.

Development and Validation of Trigger Algorithms to Identify Delays in Diagnostic Evaluation of Gastroenterological Cancer.

BACKGROUND & AIMS: Colorectal cancer (CRC) and hepatocellular cancer (HCC) are common causes of death and morbidity, and patients benefit from early detection. However, delays in follow-up of suspicious findings are common, and methods to efficiently detect such delays are needed. We developed, refined, and tested trigger algorithms that identify patients with delayed follow-up evaluation of findings suspicious of CRC or HCC. METHODS: We developed and validated two trigger algorithms that detect delays in diagnostic evaluation of CRC and HCC using laboratory, diagnosis, procedure, and referral codes from the Department of Veteran Affairs National Corporate Data Warehouse. The algorithm initially identified patients with positive test results for iron deficiency anemia or fecal immunochemical test (for CRC) and elevated α-fetoprotein results (for HCC). Our algorithm then excluded patients for whom follow-up evaluation was unnecessary, such as patients with a terminal illness or those who had already completed a follow-up evaluation within 60 days. Clinicians reviewed samples of both delayed and nondelayed records, and review data were used to calculate trigger performance. RESULTS: We applied the algorithm for CRC to 245,158 patients seen from January 1, 2013, through December 31, 2013 and identified 1073 patients with delayed follow up. In a review of 400 randomly selected records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 56.0% (95% CI, 51.0%-61.0%). We applied the algorithm for HCC to 333,828 patients seen from January 1, 2011 through December 31, 2014, and identified 130 patients with delayed follow-up. During manual review of all 130 records, we found that our algorithm identified patients with delayed follow-up with a positive predictive value of 82.3% (95% CI, 74.4%-88.2%). When we extrapolated the findings to all patients with abnormal results, the algorithm identified patients with delayed follow-up evaluation for CRC with 68.6% sensitivity (95% CI, 65.4%-71.6%) and 81.1% specificity (95% CI, 79.5%-82.6%); it identified patients with delayed follow-up evaluation for HCC with 89.1% sensitivity (95% CI, 81.8%-93.8%) and 96.5% specificity (95% CI, 94.8%-97.7%). Compared to nonselective methods, use of the algorithm reduced the number of records required for review to identify a delay by more than 99%. CONCLUSIONS: Using data from the Veterans Affairs electronic health record database, we developed an algorithm that greatly reduces the number of record reviews necessary to identify delays in follow-up evaluations for patients with suspected CRC or HCC. This approach offers a more efficient method to identify delayed diagnostic evaluation of gastroenterological cancers.

Patient perspectives on how physicians communicate diagnostic uncertainty: An experimental vignette study.

OBJECTIVE: We evaluated the effects of three different strategies for communicating diagnostic uncertainty on patient perceptions of physician competence and visit satisfaction. DESIGN/SETTING: Experimental vignette-based study design involving pediatric cases presented to a convenience sample of parents living in a large US city. PARTICIPANTS/INTERVENTION(S): Three vignettes were developed, each describing one of three different ways physicians communicated diagnostic uncertainty to parents-(i) explicit expression of uncertainty ('not sure' about diagnosis), (ii) implicit expression of uncertainty using broad differential diagnoses and (iii) implicit expression of uncertainty using 'most likely' diagnoses. Participants were randomly assigned to one of the three vignettes and then answered a 37-item web-based questionnaire. MAIN OUTCOME MEASURE(S): Outcome variables included parent-perceived technical competence of physician, trust and confidence, visit satisfaction and adherence to physician instructions. Differences between the three groups were compared using analysis of variance, followed by individual post hoc analyses with Bonferroni correction. RESULTS: Seventy-one participants completed the vignette questions. Demographic characteristics and scores on activation (parent activation measure [PAM]) and intolerance to uncertainty were similar across the three groups. Explicit expression of uncertainty was associated with lower perceived technical competence, less trust and confidence, and lower patient adherence as compared to the two groups with implicit communication. These latter two groups had comparable outcomes. CONCLUSION: Parents may react less negatively in terms of perceived competence, physician confidence and trust, and intention to adhere when diagnostic uncertainty is communicated using implicit strategies, such as using broad differential diagnoses or most likely diagnoses. Evidence-based strategies to communicate diagnostic uncertainty to patients need further development.

Electronic Detection of Delayed Test Result Follow-Up in Patients with Hypothyroidism.

BACKGROUND: Delays in following up abnormal test results are a common problem in outpatient settings. Surveillance systems that use trigger tools to identify delayed follow-up can help reduce missed opportunities in care. OBJECTIVE: To develop and test an electronic health record (EHR)-based trigger algorithm to identify instances of delayed follow-up of abnormal thyroid-stimulating hormone (TSH) results in patients being treated for hypothyroidism. DESIGN: We developed an algorithm using structured EHR data to identify patients with hypothyroidism who had delayed follow-up (>60 days) after an abnormal TSH. We then retrospectively applied the algorithm to a large EHR data warehouse within the Department of Veterans Affairs (VA), on patient records from two large VA networks for the period from January 1, 2011, to December 31, 2011. Identified records were reviewed to confirm the presence of delays in follow-up. KEY RESULTS: During the study period, 645,555 patients were seen in the outpatient setting within the two networks. Of 293,554 patients with at least one TSH test result, the trigger identified 1250 patients on treatment for hypothyroidism with elevated TSH. Of these patients, 271 were flagged as potentially having delayed follow-up of their test result. Chart reviews confirmed delays in 163 of the 271 flagged patients (PPV = 60.1%). CONCLUSIONS: An automated trigger algorithm applied to records in a large EHR data warehouse identified patients with hypothyroidism with potential delays in thyroid function test results follow-up. Future prospective application of the TSH trigger algorithm can be used by clinical teams as a surveillance and quality improvement technique to monitor and improve follow-up.

Finding Diagnostic Errors in Children Admitted to the PICU.

OBJECTIVES: To determine whether the Safer Dx Instrument, a structured tool for finding diagnostic errors in primary care, can be used to reliably detect diagnostic errors in patients admitted to a PICU. DESIGN AND SETTING: The Safer Dx Instrument consists of 11 questions to evaluate the diagnostic process and a final question to determine if diagnostic error occurred. We used the instrument to analyze four "high-risk" patient cohorts admitted to the PICU between June 2013 and December 2013. PATIENTS: High-risk cohorts were defined as cohort 1: patients who were autopsied; cohort 2: patients seen as outpatients within 2 weeks prior to PICU admission; cohort 3: patients transferred to PICU unexpectedly from an acute care floor after a rapid response and requiring vasoactive medications and/or endotracheal intubation due to decompensation within 24 hours; and cohort 4: patients transferred to PICU unexpectedly from an acute care floor after a rapid response without subsequent decompensation in 24 hours. INTERVENTIONS: Two clinicians used the instrument to independently review records in each cohort for diagnostic errors, defined as missed opportunities to make a correct or timely diagnosis. Errors were confirmed by senior expert clinicians. MEASUREMENTS AND MAIN RESULTS: Diagnostic errors were present in 26 of 214 high-risk patient records (12.1%; 95% CI, 8.2-17.5%) with the following frequency distribution: cohort 1: two of 16 (12.5%); cohort 2: one of 41 (2.4%); cohort 3: 13 of 44 (29.5%); and cohort 4: 10 of 113 (8.8%). Overall initial reviewer agreement was 93.6% (κ, 0.72). Infections and neurologic conditions were the most commonly missed diagnoses across all high-risk cohorts (16/26). CONCLUSIONS: The Safer Dx Instrument has high reliability and validity for diagnostic error detection when used in high-risk pediatric care settings. With further validation in additional clinical settings, it could be useful to enhance learning and feedback about diagnostic safety in children.

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care.

IMPORTANCE: Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement. OBJECTIVES: We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews. DESIGN: We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively). RESULTS: The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient-provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively). CONCLUSIONS: The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.

Crowdsourcing Diagnosis for Patients With Undiagnosed Illnesses: An Evaluation of CrowdMed.

BACKGROUND: Despite visits to multiple physicians, many patients remain undiagnosed. A new online program, CrowdMed, aims to leverage the "wisdom of the crowd" by giving patients an opportunity to submit their cases and interact with case solvers to obtain diagnostic possibilities. OBJECTIVE: To describe CrowdMed and provide an independent assessment of its impact. METHODS: Patients submit their cases online to CrowdMed and case solvers sign up to help diagnose patients. Case solvers attempt to solve patients' diagnostic dilemmas and often have an interactive online discussion with patients, including an exchange of additional diagnostic details. At the end, patients receive detailed reports containing diagnostic suggestions to discuss with their physicians and fill out surveys about their outcomes. We independently analyzed data collected from cases between May 2013 and April 2015 to determine patient and case solver characteristics and case outcomes. RESULTS: During the study period, 397 cases were completed. These patients previously visited a median of 5 physicians, incurred a median of US $10,000 in medical expenses, spent a median of 50 hours researching their illnesses online, and had symptoms for a median of 2.6 years. During this period, 357 active case solvers participated, of which 37.9% (132/348) were male and 58.3% (208/357) worked or studied in the medical industry. About half (50.9%, 202/397) of patients were likely to recommend CrowdMed to a friend, 59.6% (233/391) reported that the process gave insights that led them closer to the correct diagnoses, 57% (52/92) reported estimated decreases in medical expenses, and 38% (29/77) reported estimated improvement in school or work productivity. CONCLUSIONS: Some patients with undiagnosed illnesses reported receiving helpful guidance from crowdsourcing their diagnoses during their difficult diagnostic journeys. However, further development and use of crowdsourcing methods to facilitate diagnosis requires long-term evaluation as well as validation to account for patients' ultimate correct diagnoses.

Diagnostic errors related to acute abdominal pain in the emergency department.

OBJECTIVE: Diagnostic errors in the emergency department (ED) are harmful and costly. We reviewed a selected high-risk cohort of patients presenting to the ED with abdominal pain to evaluate for possible diagnostic errors and associated process breakdowns. DESIGN: We conducted a retrospective chart review of ED patients >18 years at an urban academic hospital. A computerised 'trigger' algorithm identified patients possibly at high risk for diagnostic errors to facilitate selective record reviews. The trigger determined patients to be at high risk because they: (1) presented to the ED with abdominal pain, and were discharged home and (2) had a return ED visit within 10 days that led to a hospitalisation. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available during the first ED visit, regardless of patient harm, and included errors that involved both ED and non-ED providers. Errors were determined by two independent record reviewers followed by team consensus in cases of disagreement. RESULTS: Diagnostic errors occurred in 35 of 100 high-risk cases. Over two-thirds had breakdowns involving the patient-provider encounter (most commonly history-taking or ordering additional tests) and/or follow-up and tracking of diagnostic information (most commonly follow-up of abnormal test results). The most frequently missed diagnoses were gallbladder pathology (n=10) and urinary infections (n=5). CONCLUSIONS: Diagnostic process breakdowns in ED patients with abdominal pain most commonly involved history-taking, ordering insufficient tests in the patient-provider encounter and problems with follow-up of abnormal test results.

Development and Validation of Electronic Health Record-based Triggers to Detect Delays in Follow-up of Abnormal Lung Imaging Findings.

Purpose To develop an electronic health record (EHR)-based trigger algorithm to identify delays in follow-up of patients with imaging results that are suggestive of lung cancer and to validate this trigger on retrospective data. Materials and Methods The local institutional review board approved the study. A "trigger" algorithm was developed to automate the detection of delays in diagnostic evaluation of chest computed tomographic (CT) images and conventional radiographs that were electronically flagged by reviewing radiologists as being "suspicious for malignancy." The trigger algorithm was developed through literature review and expert input. It included patients who were alive and 40-70 years old, and it excluded instances in which appropriate timely follow-up (defined as occurring within 30 days) was detected (eg, pulmonary visit) or when follow-up was unnecessary (eg, in patients with a terminal illness). The algorithm was iteratively applied to a retrospective test cohort in an EHR data warehouse at a large Veterans Affairs facility, and manual record reviews were used to validate each individual criterion. The final algorithm aimed at detecting an absence of timely follow-up was retrospectively applied to an independent validation cohort to determine the positive predictive value (PPV). Trigger performance, time to follow-up, reasons for lack of follow-up, and cancer outcomes were analyzed and reported by using descriptive statistics. Results The trigger algorithm was retrospectively applied to the records of 89 168 patients seen between January 1, 2009, and December 31, 2009. Of 538 records with an imaging report that was flagged as suspicious for malignancy, 131 were identified by the trigger as being high risk for delayed diagnostic evaluation. Manual chart reviews confirmed a true absence of follow-up in 75 cases (trigger PPV of 57.3% for detecting evaluation delays), of which four received a diagnosis of primary lung cancer within the subsequent 2 years. Conclusion EHR-based triggers can be used to identify patients with suspicious imaging findings in whom follow-up diagnostic evaluation was delayed. (©) RSNA, 2015.

Diagnostic error in mental health: a review.

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Medical Home Implementation and Follow-Up of Cancer-Related Abnormal Test Results in the Veterans Health Administration.

Importance: Lack of timely follow-up of cancer-related abnormal test results can lead to delayed or missed diagnoses, adverse cancer outcomes, and substantial cost burden for patients. Care delivery models, such as the Veterans Affairs' (VA) Patient-Aligned Care Team (PACT), which aim to improve patient-centered care coordination, could potentially also improve timely follow-up of abnormal test results. PACT was implemented nationally in the VA between 2010 and 2012. Objective: To evaluate the long-term association between PACT implementation and timely follow-up of abnormal test results related to the diagnosis of 5 different cancers. Design, Setting, and Participants: This multiyear retrospective cohort study used 14 years of VA data (2006-2019), which were analyzed using panel data-based random-effects linear regressions. The setting included all VA clinics and facilities. The participants were adult patients who underwent diagnostic testing related to 5 different cancers and had abnormal test results. Data extraction and statistical analyses were performed from September 2021 to December 2023. Exposure: Calendar years denoting preperiods and postperiods of PACT implementation, and the PACT Implementation Progress Index Score denoting the extent of implementation in each VA clinic and facility. Main Outcome and Measure: Percentage of potentially missed timely follow-ups of abnormal test results. Results: This study analyzed 6 data sets representing 5 different types of cancers. During the initial years of PACT implementation (2010 to 2013), percentage of potentially missed timely follow-ups decreased between 3 to 7 percentage points for urinalysis suggestive of bladder cancer, 12 to 14 percentage points for mammograms suggestive of breast cancer, 19 to 22 percentage points for fecal tests suggestive of colorectal cancer, and 6 to 13 percentage points for iron deficiency anemia laboratory tests suggestive of colorectal cancer, with no statistically significant changes for α-fetoprotien tests and lung cancer imaging. However, these beneficial reductions were not sustained over time. Better PACT implementation scores were associated with a decrease in potentially missed timely follow-up percentages for urinalysis (0.3-percentage point reduction [95% CI, -0.6 to -0.1] with 1-point increase in the score), and laboratory tests suggestive of iron deficiency anemia (0.5-percentage point reduction [95% CI,-0.8 to -0.2] with 1-point increase in the score). Conclusions and Relevance: This cohort study found that implementation of PACT in the VA was associated with a potential short-term improvement in the quality of follow-up for certain test results. Additional multifaceted sustained interventions to reduce missed test results are required to prevent care delays.

Cancer Evaluations During the COVID-19 Pandemic: An Observational Study Using National Veterans Affairs Data.

INTRODUCTION: Fewer cancer diagnoses have been made during the COVID-19 pandemic. Pandemic-related delays in cancer diagnosis could occur from limited access to care or patient evaluation delays (e.g., delayed testing after abnormal results). Follow-up of abnormal test results warranting evaluation for cancer was examined before and during the pandemic. METHODS: Electronic trigger algorithms were applied to the Department of Veterans Affairs electronic health record data to assess follow-up of abnormal test results before (March 10, 2019-March 7, 2020) and during (March 8, 2020-March 6, 2021) the pandemic. RESULTS: Electronic triggers were applied to 8,021,406 veterans' electronic health records to identify follow-up delays for abnormal results warranting evaluation for 5 cancers: bladder (urinalysis with high-grade hematuria), breast (abnormal mammograms), colorectal (positive fecal occult blood tests/fecal immunochemical tests or results consistent with iron deficiency anemia), liver (elevated alpha-fetoprotein), and lung (chest imaging suggestive of malignancy) cancers. Between prepandemic and pandemic periods, test quantities decreased by 12.6%-27.8%, and proportions of abnormal results lacking follow-up decreased for urinalyses (-0.8%), increased for fecal occult blood tests/fecal immunochemical test (+2.3%) and chest imaging (+1.8%), and remained constant for others. Follow-up times decreased for most tests; however, control charts suggested increased delays at 2 stages: early (pandemic beginning) for urinalyses, mammograms, fecal occult blood tests/fecal immunochemical test, iron deficiency anemia, and chest imaging and late (30-45 weeks into pandemic) for mammograms, fecal occult blood tests/fecal immunochemical test, and iron deficiency anemia. CONCLUSIONS: Although early pandemic delays in follow-up may have led to reduced cancer rates, the significant decrease in tests performed is likely a large driver of these reductions. Future emergency preparedness efforts should bolster essential follow-up and testing procedures to facilitate timely cancer diagnosis.

Electronic Co-design (ECO-design) Workshop for Increasing Clinician Participation in the Design of Health Services Interventions: Participatory Design Approach.

BACKGROUND: Participation from clinician stakeholders can improve the design and implementation of health care interventions. Participatory design methods, especially co-design methods, comprise stakeholder-led design activities that are time-consuming. Competing work demands and increasing workloads make clinicians' commitments to typical participatory methods even harder. The COVID-19 pandemic further exacerbated barriers to clinician participation in such interventions. OBJECTIVE: The aim of this study was to explore a web-based participatory design approach to conduct economical, electronic co-design (ECO-design) workshops with primary care clinicians. METHODS: We adapted traditional in-person co-design workshops to web-based delivery and adapted co-design workshop series to fit within a single 1-hour session. We applied the ECO-design workshop approach to codevelop feedback interventions regarding abnormal test result follow-up in primary care. We conducted ECO-design workshops with primary care clinicians at a medical center in Southern Texas, using videoconferencing software. Each workshop focused on one of three types of feedback interventions: conversation guide, email template, and dashboard prototype. We paired electronic materials and software features to facilitate participant interactions, prototyping, and data collection. The workshop protocol included four main activities: problem identification, solution generation, prototyping, and debriefing. Two facilitators were assigned to each workshop and one researcher resolved technical problems. After the workshops, our research team met to debrief and evaluate workshops. RESULTS: A total of 28 primary care clinicians participated in our ECO-design workshops. We completed 4 parallel workshops, each with 5-10 participants. We conducted traditional analyses and generated a clinician persona (ie, representative description) and user interface prototypes. We also formulated recommendations for future ECO-design workshop recruitment, technology, facilitation, and data collection. Overall, our adapted workshops successfully enabled primary care clinicians to participate without increasing their workload, even during a pandemic. CONCLUSIONS: ECO-design workshops are viable, economical alternatives to traditional approaches. This approach fills a need for efficient methods to involve busy clinicians in the design of health care interventions.

Patient and clinician experiences of uncertainty in the diagnostic process: Current understanding and future directions.

OBJECTIVE: Uncertainty occurs throughout the diagnostic process and must be managed to facilitate accurate and timely diagnoses and treatments. Better characterization of uncertainty can inform strategies to manage it more effectively in clinical practice. We provide a comprehensive overview of current literature on diagnosis-related uncertainty describing (1) where patients and clinicians experience uncertainty within the diagnostic process, (2) how uncertainty affects the diagnostic process, (3) roots of uncertainty related to probability/risk, ambiguity, or complexity, and (4) strategies to manage uncertainty. DISCUSSION: Each diagnostic process step involves uncertainty, including patient engagement with the healthcare system; information gathering, interpretation, and integration; formulating working diagnoses; and communicating diagnoses to patients. General management strategies include acknowledging uncertainty, obtaining more contextual information from patients (e.g., gathering occupations and family histories), creating diagnostic safety nets (e.g., informing patients what red flags to look for), engaging in worst case/best case scenario planning, and communicating diagnostic uncertainty to patients, families, and colleagues. Potential strategies tailored to various aspects of diagnostic uncertainty are also outlined. CONCLUSION: Scientific knowledge on diagnostic uncertainty, while previously elusive, is now becoming more clearly defined. Next steps include research to evaluate relationships between management and communication of diagnostic uncertainty and improved patient outcomes.

A Program to Provide Clinicians with Feedback on Their Diagnostic Performance in a Learning Health System.

PROBLEM: Reducing diagnostic errors requires improving both systems and individual clinical reasoning. One strategy to achieve diagnostic excellence is learning from feedback. However, clinicians remain uncomfortable receiving feedback on their diagnostic performance. Thus, a team of researchers and clinical leaders aimed to develop and implement a diagnostic performance feedback program for learning that mitigates potential clinician discomfort. APPROACH: The program was developed as part of a larger project to create a learning health system around diagnostic safety at Geisinger, a large, integrated health care system in rural Pennsylvania. Steps included identifying potential missed opportunities in diagnosis (MODs) from various sources (for example, risk management, clinician reports, patient complaints); confirming MODs through chart review; and having trained facilitators provide feedback to clinicians about MODs as learning opportunities. The team developed a guide for facilitators to conduct effective diagnostic feedback sessions and surveyed facilitators and recipients about their experiences and perceptions of the feedback sessions. OUTCOMES: 28 feedback sessions occurred from January 2019 to June 2020, involving MODs from emergency medicine, primary care, and hospital medicine. Most facilitators (90.6% [29/32]) reported that recipients were receptive to learning and discussing MODs. Most recipients reported that conversations were constructive and nonpunitive (83.3% [25/30]) and allowed them to take concrete steps toward improving diagnosis (76.7% [23/30]). Both groups believed discussions would improve future diagnostic safety (93.8% [30/32] and 70.0% [21/30], respectively). KEY INSIGHTS AND NEXT STEPS: An institutional program was developed and implemented to deliver diagnostic performance feedback. Such a program may facilitate learning and improvement to reduce MODs. Future efforts should assess long-term effects on diagnostic performance and patient outcomes.