Creating a Learning Health System for Improving Diagnostic Safety: Pragmatic Insights from US Health Care Organizations.

OBJECTIVE: To identify challenges and pragmatic strategies for improving diagnostic safety at an organizational level using concepts from learning health systems METHODS: We interviewed 32 safety leaders across the USA on how their organizations approach diagnostic safety. Participants were recruited through email and represented geographically diverse academic and non-academic settings. The interview included questions on culture of reporting and learning from diagnostic errors; data gathering and analysis activities; diagnostic training and educational activities; and engagement of clinical leadership, staff, patients, and families in diagnostic safety activities. We conducted an inductive content analysis of interview transcripts and two reviewers coded all data. RESULTS: Of 32 participants, 12 reported having a specific program to address diagnostic errors. Multiple barriers to implement diagnostic safety activities emerged: serious concerns about psychological safety associated with diagnostic error; lack of infrastructure for measurement, monitoring, and improvement activities related to diagnosis; lack of leadership investment, which was often diverted to competing priorities related to publicly reported measures or other incentives; and lack of dedicated teams to work on diagnostic safety. Participants provided several strategies to overcome barriers including adapting trigger tools to identify safety events, engaging patients in diagnostic safety, and appointing dedicated diagnostic safety champions. CONCLUSIONS: Several foundational building blocks related to learning health systems could inform organizational efforts to reduce diagnostic error. Promoting an organizational culture specific to diagnostic safety, using science and informatics to improve measurement and analysis, leadership incentives to build institutional capacity to address diagnostic errors, and patient engagement in diagnostic safety activities can enable progress.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps.

BACKGROUND: Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm. OBJECTIVE: As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff. DESIGN: We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety. SETTING: Twelve VA facilities across the USA. PARTICIPANTS: Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership. APPROACH: We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement. RESULTS: We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme. CONCLUSIONS: Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study.

BACKGROUND: Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful. OBJECTIVE: The study aimed to examine patients' experiences using an artificial intelligence (AI)-assisted online symptom checker. METHODS: An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected. RESULTS: A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool's results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool's results (20/34, 59% vs 22/69, 32%; P=.04). CONCLUSIONS: Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.

Understanding the impact of gendered roles on the experiences of infertility amongst men and women in Punjab.

While infertility is a global challenge for millions of couples, low income countries have particularly high rates, of up to 30%. Infertility in these contexts is not limited to its clinical definition but is a socially constructed notion with varying definitions. In highly pronatalistic and patriarchal societies like Pakistan, women bear the brunt of the social, emotional and physical consequences of childlessness. While the often harsh consequences of childlessness for Pakistani women have been widely documented, there is a dearth of exploration into the ways in which prescribed gender roles inform the experiences of childlessness among Pakistani women and men. The aim of this study was to explore and compare how gender ideologies, values and expectations shape women's and men's experiences of infertility in Pakistan. Using an interpretive descriptive approach, in-depth interviews were conducted with 12 women and 8 men experiencing childlessness in Punjab, Pakistan from April to May 2008. Data analysis was thematic and inductive based on the principles of content analysis. The experience of infertility for men and women is largely determined by their prescribed gender roles. Childlessness weakened marital bonds with gendered consequences. For women, motherhood is not only a source of status and power, it is the only avenue for women to ensure their marital security. Weak marital ties did not affect men's social identity, security or power. Women also face harsher psychosocial, social, emotional and physical consequences of childlessness than men. They experienced abuse, exclusion and stigmatization at the couple, household and societal level, while men only experienced minor taunting from friends. Women unceasingly sought invasive infertility treatments, while most men assumed there was nothing wrong with themselves. This study highlights the ways in which gender roles and norms shape the experiences associated with involuntary childlessness for men and women in Punjab, Pakistan. The insight obtained into the range of experiences can potentially contribute to deeper understanding of the social construction of infertility and childlessness in pronatalistic and patriarchal societies as well as the ways in which gender ideologies operationalise to marginalise women.

Developing the Safer Dx Checklist of Ten Safety Recommendations for Health Care Organizations to Address Diagnostic Errors.

BACKGROUND: Most health care organizations (HCOs) find diagnostic errors hard to address. The research team developed a checklist (the Safer Dx Checklist) of 10 high-priority safety practices HCOs can use to conduct a proactive risk assessment to address diagnostic error. METHODS: First, the team identified potential practices based on reviews of recent literature, reports by national and international organizations, and interviews with quality/safety leaders. Then a Delphi panel was conducted, followed by an online expert panel, to prioritize 10 practices. The prioritization process considered impact on safety and feasibility of practice implementation within a one- to three-year time frame. Finally, cognitive walkthroughs were conducted for a face-validity check with end users. The team also conducted content analysis in each step to look for themes that influenced prioritization or checklist implementation. RESULTS: A total of 71 practices for prioritization were identified through the Delphi panel of 28 experts; 65% of participants reached consensus on 28 practices. A multidisciplinary panel of 10 experts helped prioritize and refine the top 10 practices, which were then developed into a checklist paired with implementation guidance. Practices included themes related to creating organizational and leadership accountability for improving diagnosis, including patients in diagnostic safety work, and developing and implementing organizational infrastructure for measurement and improvement activities. Qualitative analysis revealed insights for implementation. End users at three different HCOs helped refine implementation guidance for the checklist. CONCLUSION: The researchers identified 10 safety practices to help organizations conduct a proactive, systematic assessment of risks to timely and accurate diagnosis. The Safer Dx Checklist can enable HCOs to begin implementing strategies to address diagnostic error.

Development and Usability Testing of the Agency for Healthcare Research and Quality Common Formats to Capture Diagnostic Safety Events.

OBJECTIVES: A lack of consensus around definitions and reporting standards for diagnostic errors limits the extent to which healthcare organizations can aggregate, analyze, share, and learn from these events. In response to this problem, the Agency for Healthcare Research and Quality (AHRQ) began the development of the Common Formats for Event Reporting for Diagnostic Safety Events (CFER-DS). We conducted a usability assessment of the draft CFER-DS to inform future revision and implementation. METHODS: We recruited a purposive sample of quality and safety personnel working in 8 U.S. healthcare organizations. Participants were invited to use the CFER-DS to simulate reporting for a minimum of 5 cases of diagnostic safety events and then provide written and verbal qualitative feedback. Analysis focused on participants' perceptions of content validity, ease of use, and potential for implementation. RESULTS: Estimated completion time was 30 to 90 minutes per event. Participants shared generally positive feedback about content coverage and item clarity but identified reporter burden as a potential concern. Participants also identified opportunities to clarify several conceptual definitions, ensure applicability across different care settings, and develop guidance to operationalize use of CFER-DS. Findings led to refinement of content and supplementary materials to facilitate implementation. CONCLUSIONS: Standardized definitions of diagnostic safety events and reporting standards for contextual information and contributing factors can help capture and analyze diagnostic safety events. In addition to usability testing, additional feedback from the field will ensure that AHRQ's CFER-DS is useful to a broad range of users for learning and safety improvement.

Electronic Co-design (ECO-design) Workshop for Increasing Clinician Participation in the Design of Health Services Interventions: Participatory Design Approach.

BACKGROUND: Participation from clinician stakeholders can improve the design and implementation of health care interventions. Participatory design methods, especially co-design methods, comprise stakeholder-led design activities that are time-consuming. Competing work demands and increasing workloads make clinicians' commitments to typical participatory methods even harder. The COVID-19 pandemic further exacerbated barriers to clinician participation in such interventions. OBJECTIVE: The aim of this study was to explore a web-based participatory design approach to conduct economical, electronic co-design (ECO-design) workshops with primary care clinicians. METHODS: We adapted traditional in-person co-design workshops to web-based delivery and adapted co-design workshop series to fit within a single 1-hour session. We applied the ECO-design workshop approach to codevelop feedback interventions regarding abnormal test result follow-up in primary care. We conducted ECO-design workshops with primary care clinicians at a medical center in Southern Texas, using videoconferencing software. Each workshop focused on one of three types of feedback interventions: conversation guide, email template, and dashboard prototype. We paired electronic materials and software features to facilitate participant interactions, prototyping, and data collection. The workshop protocol included four main activities: problem identification, solution generation, prototyping, and debriefing. Two facilitators were assigned to each workshop and one researcher resolved technical problems. After the workshops, our research team met to debrief and evaluate workshops. RESULTS: A total of 28 primary care clinicians participated in our ECO-design workshops. We completed 4 parallel workshops, each with 5-10 participants. We conducted traditional analyses and generated a clinician persona (ie, representative description) and user interface prototypes. We also formulated recommendations for future ECO-design workshop recruitment, technology, facilitation, and data collection. Overall, our adapted workshops successfully enabled primary care clinicians to participate without increasing their workload, even during a pandemic. CONCLUSIONS: ECO-design workshops are viable, economical alternatives to traditional approaches. This approach fills a need for efficient methods to involve busy clinicians in the design of health care interventions.

Use of patient complaints to identify diagnosis-related safety concerns: a mixed-method evaluation.

BACKGROUND: Patient complaints are associated with adverse events and malpractice claims but underused in patient safety improvement. OBJECTIVE: To systematically evaluate the use of patient complaint data to identify safety concerns related to diagnosis as an initial step to using this information to facilitate learning and improvement. METHODS: We reviewed patient complaints submitted to Geisinger, a large healthcare organisation in the USA, from August to December 2017 (cohort 1) and January to June 2018 (cohort 2). We selected complaints more likely to be associated with diagnostic concerns in Geisinger's existing complaint taxonomy. Investigators reviewed all complaint summaries and identified cases as 'concerning' for diagnostic error using the National Academy of Medicine's definition of diagnostic error. For all 'concerning' cases, a clinician-reviewer evaluated the associated investigation report and the patient's medical record to identify any missed opportunities in making a correct or timely diagnosis. In cohort 2, we selected a 10% sample of 'concerning' cases to test this smaller pragmatic sample as a proof of concept for future organisational monitoring. RESULTS: In cohort 1, we reviewed 1865 complaint summaries and identified 177 (9.5%) concerning reports. Review and analysis identified 39 diagnostic errors. Most were categorised as 'Clinical Care issues' (27, 69.2%), defined as concerns/questions related to the care that is provided by clinicians in any setting. In cohort 2, we reviewed 2423 patient complaint summaries and identified 310 (12.8%) concerning reports. The 10% sample (n=31 cases) contained five diagnostic errors. Qualitative analysis of cohort 1 cases identified concerns about return visits for persistent and/or worsening symptoms, interpersonal issues and diagnostic testing. CONCLUSIONS: Analysis of patient complaint data and corresponding medical record review identifies patterns of failures in the diagnostic process reported by patients and families. Health systems could systematically analyse available data on patient complaints to monitor diagnostic safety concerns and identify opportunities for learning and improvement.