The Limits of Clinician Vigilance as an AI Safety Bulwark.

This Viewpoint examines the potential problems of clinician reliance on the use of artificial intelligence (AI) in health care and offers suggestions on how AI could be designed to promote clinician vigilance.

Primary Care Physicians' Satisfaction With Interoperable Health Information Technology.

Importance: Enabling widespread interoperability-the ability of health information technology systems to exchange information and to use that information without special effort-is a primary focus of public policy on health information technology. More information on clinicians' experience using that technology can serve as one measure of the impact of that policy. Objective: To assess primary care physician perspectives on the state of interoperability. Design, Setting, and Participants: A cross-sectional survey of family medicine physicians in the US was conducted from December 12, 2021, to October 12, 2022. A sample of family medicine physicians who completed the Continuous Certification Questionnaire (CCQ), a required part of the American Board of Family Medicine certification process, which has a 100% response rate, were invited to participate. Main Outcomes and Measures: Eighteen items on the CCQ assessed experience accessing and using various information from outside organizations, including medications, immunizations, and allergies. Results: A total of 2088 physicians (1053 women [50%]; age reported categorically as either ≥50 years or <50 years) completed the CCQ interoperability questions in 2022. Of these respondents, 35% practiced in hospital or health system-owned practices, while 27% practiced in independently owned practices. Eleven percent were very satisfied with their ability to electronically access all 10 types of information from outside organizations included on the questionnaire, and a mean of 70% were at least somewhat satisfied. A total of 23% of family medicine physicians reported information from outside organizations was very easy to use, and an additional 65% reported that information was somewhat easy to use. Only 8% reported that information from different electronic health record (EHR) developers' products was very easy to use compared with 38% who reported information from the same EHR developer's product was very easy to use. Conclusions and Relevance: This survey study of family medicine physicians found modest and uneven improvement in physicians' experience with interoperability. These findings suggest that substantial heterogeneity in satisfaction by information type, source of information, EHR, practice type, ownership, and patient population necessitates diverse policy and strategies to improve interoperability.

Alternative Payment Models and Patient-Reported Quality of Preparation for Discharge: A Retrospective Longitudinal Study.

Preparing patients for posthospital care may improve readmission risk. Alternative payment models (APMs) incent hospitals to reduce readmissions by tying payment to outcomes. The impact of APMs on preparation for discharge is not well understood. We assessed whether patient-reported preparation for posthospital care was associated with reduced readmissions, and whether APM participation was associated with improved preparation for posthospital care. We used mixed-effects regression on retrospective (2013-2017) observational data for 2685 U.S. hospitals. We measured patient-reported preparation for posthospital care using the 3-Item Care Transition Measure and readmission using 30-day all-cause risk-adjusted readmissions from Hospital Compare. Participation in accountable care organizations (ACOs), Medical Homes, and Medicare's Bundled Payments for Care Improvement program was obtained from Medicare, the American Hospital Association's Annual Survey, and Leavitt Partner's ACO database. We found that APMs are not associated with improved preparation for posthospital care, even though it was associated with reduced readmissions (Marginal Effect: -0.012 percentage points). This may be because hospitals are not investing in patient engagement. This study has limited insight into causality and reduced generalizability among smaller, rural, and non-teaching hospitals.

Public perspectives on the use of different data types for prediction in healthcare.

OBJECTIVE: Understand public comfort with the use of different data types for predictive models. MATERIALS AND METHODS: We analyzed data from a national survey of US adults (n = 1436) fielded from November to December 2021. For three categories of data (identified using factor analysis), we use descriptive statistics to capture comfort level. RESULTS: Public comfort with data use for prediction is low. For 13 of 15 data types, most respondents were uncomfortable with that data being used for prediction. In factor analysis, 15 types of data grouped into three categories based on public comfort: (1) personal characteristic data, (2) health-related data, and (3) sensitive data. Mean comfort was highest for health-related data (2.45, SD 0.84, range 1-4), followed by personal characteristic data (2.36, SD 0.94), and sensitive data (1.88, SD 0.77). Across these categories, we observe a statistically significant positive relationship between trust in health systems' use of patient information and comfort with data use for prediction. DISCUSSION: Although public trust is recognized as important for the sustainable expansion of predictive tools, current policy does not reflect public concerns. Low comfort with data use for prediction should be addressed in order to prevent potential negative impacts on trust in healthcare. CONCLUSION: Our results provide empirical evidence on public perspectives, which are important for shaping the use of predictive models. Findings demonstrate a need for realignment of policy around the sensitivity of non-clinical data categories.

A national survey of digital health company experiences with electronic health record application programming interfaces.

OBJECTIVES: This study sought to capture current digital health company experiences integrating with electronic health records (EHRs), given new federally regulated standards-based application programming interface (API) policies. MATERIALS AND METHODS: We developed and fielded a survey among companies that develop solutions enabling human interaction with an EHR API. The survey was developed by the University of California San Francisco in collaboration with the Office of the National Coordinator for Health Information Technology, the California Health Care Foundation, and ScaleHealth. The instrument contained questions pertaining to experiences with API integrations, barriers faced during API integrations, and API-relevant policy efforts. RESULTS: About 73% of companies reported current or previous use of a standards-based EHR API in production. About 57% of respondents indicated using both standards-based and proprietary APIs to integrate with an EHR, and 24% worked about equally with both APIs. Most companies reported use of the Fast Healthcare Interoperability Resources standard. Companies reported that standards-based APIs required on average less burden than proprietary APIs to establish and maintain. However, companies face barriers to adopting standards-based APIs, including high fees, lack of realistic clinical testing data, and lack of data elements of interest or value. DISCUSSION: The industry is moving toward the use of standardized APIs to streamline data exchange, with a majority of digital health companies using standards-based APIs to integrate with EHRs. However, barriers persist. CONCLUSION: A large portion of digital health companies use standards-based APIs to interoperate with EHRs. Continuing to improve the resources for digital health companies to find, test, connect, and use these APIs "without special effort" will be crucial to ensure future technology robustness and durability.

National Trends in Billing Secure Messages as E-Visits.

This study assesses US trends in e-visit billing using national all-payer claims.

How patients distinguish between clinical and administrative predictive models in health care.

OBJECTIVES: To understand patient perceptions of specific applications of predictive models in health care. STUDY DESIGN: Original, cross-sectional national survey. METHODS: We conducted a national online survey of US adults with the National Opinion Research Center from November to December 2021. Measures of internal consistency were used to identify how patients differentiate between clinical and administrative predictive models. Multivariable logistic regressions were used to identify relationships between comfort with various types of predictive models and patient demographics, perceptions of privacy protections, and experiences in the health care system. RESULTS: A total of 1541 respondents completed the survey. After excluding observations with missing data for the variables of interest, the final analytic sample was 1488. We found that patients differentiate between clinical and administrative predictive models. Comfort with prediction of bill payment and missed appointments was especially low (21.6% and 36.6%, respectively). Comfort was higher with clinical predictive models, such as predicting stroke in an emergency (55.8%). Experiences of discrimination were significant negative predictors of comfort with administrative predictive models. Health system transparency around privacy policies was a significant positive predictor of comfort with both clinical and administrative predictive models. CONCLUSIONS: Patients are more comfortable with clinical applications of predictive models than administrative ones. Privacy protections and transparency about how health care systems protect patient data may facilitate patient comfort with these technologies. However, larger inequities and negative experiences in health care remain important for how patients perceive administrative applications of prediction.

Citizens' Access to Online Health Information - An International Survey of IMIA Member Countries.

Citizens' access to their online health information is pivotal. Therefore, this study examines citizens' access to their online health information across countries and healthcare settings. The study is based on a survey design targeting the 98 IMIA representatives of the national societies. Results indicate that Test results and Medications are the two types of online information that citizens in most cases have access to. Ten countries provide citizens access to all the different types of information included in the study. That relatively few countries provide citizens access to all the included types of online health information underscores the importance of continuous emphasis on accessibility and research within this field.

Structured and unstructured social risk factor documentation in the electronic health record underestimates patients' self-reported risks.

OBJECTIVES: National attention has focused on increasing clinicians' responsiveness to the social determinants of health, for example, food security. A key step toward designing responsive interventions includes ensuring that information about patients' social circumstances is captured in the electronic health record (EHR). While prior work has assessed levels of EHR "social risk" documentation, the extent to which documentation represents the true prevalence of social risk is unknown. While no gold standard exists to definitively characterize social risks in clinical populations, here we used the best available proxy: social risks reported by patient survey. MATERIALS AND METHODS: We compared survey results to respondents' EHR social risk documentation (clinical free-text notes and International Statistical Classification of Diseases and Related Health Problems [ICD-10] codes). RESULTS: Surveys indicated much higher rates of social risk (8.2%-40.9%) than found in structured (0%-2.0%) or unstructured (0%-0.2%) documentation. DISCUSSION: Ideally, new care standards that include incentives to screen for social risk will increase the use of documentation tools and clinical teams' awareness of and interventions related to social adversity, while balancing potential screening and documentation burden on clinicians and patients. CONCLUSION: EHR documentation of social risk factors currently underestimates their prevalence.

Guidance for reporting analyses of metadata on electronic health record use.

INTRODUCTION: Research on how people interact with electronic health records (EHRs) increasingly involves the analysis of metadata on EHR use. These metadata can be recorded unobtrusively and capture EHR use at a scale unattainable through direct observation or self-reports. However, there is substantial variation in how metadata on EHR use are recorded, analyzed and described, limiting understanding, replication, and synthesis across studies. RECOMMENDATIONS: In this perspective, we provide guidance to those working with EHR use metadata by describing 4 common types, how they are recorded, and how they can be aggregated into higher-level measures of EHR use. We also describe guidelines for reporting analyses of EHR use metadata-or measures of EHR use derived from them-to foster clarity, standardization, and reproducibility in this emerging and critical area of research.

Developing electronic health record-based measures of the 4Ms to support implementation and evidence generation for Age-Friendly Health Systems.

BACKGROUND: To support implementation of the 4Ms framework and more rigorous evidence of 4Ms impact, we translated Institute for Healthcare Improvement's (IHI's) recommended 4Ms routine care practices into electronic health record-based, encounter-level adherence measures and then implemented measures at a large academic medical center. METHODS: We started with the 19 care practices in IHI's 4Ms implementation guide and developed encounter-level adherence measures using structured EHR data. We also developed overall 4Ms-level and M-level composite measures. Next, we operationalized measures at UCSF Health-an academic medical center that has implemented the 4Ms using the IHI guide. We identified UCSF Health patients who should have received 4Ms care during their inpatient admission (19,335 individuals 65 years and older with an admission between January 1, 2019 and December 31, 2021), then implemented the individual measures and composite measures (all at the encounter level) using Epic EHR data. We focused on 4Ms inpatient care processes, but similar approaches can be followed for ambulatory, post-acute, and other settings. RESULTS: We developed 18 EHR-based measures that captured all IHI care practices, 16 of which could be implemented using UCSF Health EHR data. For example, the EHR-based measure for the Medication care practice "deprescribe high risk medications" was measured using EHR data as "Patient had no previously existing prescriptions for high-risk medications OR patient had ≥1 previously existing prescriptions for high-risk medications deprescribed during the encounter," and 29.5% of UCSF Health encounters met this measure. For composite measures, on average, UCSF Health encounters had 61.1% adherence to the 4Ms (SD = 14.4%), with the lowest average adherence to What Matters (50.9%; SD = 44.3%) and the highest for Mentation (68.4%; SD = 13.4%). CONCLUSIONS: It is feasible to construct encounter-level measures of 4Ms adherence using EHR data and derive insights to guide ongoing implementation efforts. Future efforts should refine measures based on assessments of reliability and validity.

Electronic connectivity between hospital pairs: impact on emergency department-related utilization.

OBJECTIVE: To use more precise measures of which hospitals are electronically connected to determine whether health information exchange (HIE) is associated with lower emergency department (ED)-related utilization. MATERIALS AND METHODS: We combined 2018 Medicare fee-for-service claims to identify beneficiaries with 2 ED encounters within 30 days, and Definitive Healthcare and AHA IT Supplement data to identify hospital participation in HIE networks (HIOs and EHR vendor networks). We determined whether the 2 encounters for the same beneficiary occurred at: the same organization, different organizations connected by HIE, or different organizations not connected by HIE. Outcomes were: (1) whether any repeat imaging occurred during the second ED visit; (2) for beneficiaries with a treat-and-release ED visit followed by a second ED visit, whether they were admitted to the hospital after the second visit; (3) for beneficiaries discharged from the hospital followed by an ED visit, whether they were admitted to the hospital. RESULTS: In adjusted mixed effects models, for all outcomes, beneficiaries returning to the same organization had significantly lower utilization compared to those going to different organizations. Comparing only those going to different organizations, HIE was not associated with lower levels of repeat imaging. HIE was associated with lower likelihood of hospital admission following a treat-and-release ED visit (1.83 percentage points [-3.44 to -0.21]) but higher likelihood of admission following hospital discharge (2.78 percentage points [0.48-5.08]). DISCUSSION: Lower utilization for beneficiaries returning to the same organization could reflect better access to information or other factors such as aligned incentives. CONCLUSION: HIE is not consistently associated with utilization outcomes reflecting more coordinated care in the ED setting.

Changes in Physician Electronic Health Record Use With the Expansion of Telemedicine.

Importance: Understanding the drivers of electronic health record (EHR) burden, including EHR time and patient messaging, may directly inform strategies to address physician burnout. Given the COVID-19-induced expansion of telemedicine-now used for a substantial proportion of ambulatory encounters-its association with EHR burden should be evaluated. Objective: To measure the association of the telemedicine expansion with time spent working in the EHR and with patient messaging among ambulatory physicians before and after the onset of the COVID-19 pandemic. Design, Setting, and Participants: This longitudinal cohort study analyzed weekly EHR metadata of ambulatory physicians at UCSF Health, a large academic medical center. The same EHR measures were compared for 1 year before the COVID-19 pandemic (August 2018-September 2019) with the same period 1 year after its onset (August 2020-September 2021). Multivariable regression models evaluating the association between level of telemedicine use and EHR use were then assessed after the onset of the pandemic. The sample included all physician-weeks with at least 1 scheduled half-day clinic in the 11 largest ambulatory specialties at UCSF Health. Data analyses were performed from March 1, 2022, through July 1, 2023. Exposures: Physicians' weekly modality mix of either entirely face-to-face visits, mixed modalities, or entirely telemedicine. Main Outcomes and Measures: The EHR time during and outside of patient scheduled hours (PSHs), time spent documenting (normalized per 8 PSHs), and electronic messages sent to and received from patients. Results: The study sample included 1052 physicians (437 [41.5%] men and 615 [58.5%] women) during 115 weeks, which provided 35 697 physician-week observations. Comparing the period before to the period after pandemic onset showed that physician time spent working in the EHR during PSHs increased from 4.53 to 5.46 hours per 8 PSH (difference, 0.93; 95% CI, 0.87-0.98; P < 0.001); outside of PSHs, increased from 4.29 to 5.34 hours (difference, 1.04; 95% CI, 0.95-1.14; P < 0.001); and time documenting during and outside of PSHs increased from 6.35 to 8.18 hours (difference, 1.83; 95% CI, 1.72-1.94; P < 0.001). Mean weekly messages received from patients increased from 16.76 to 30.33, and messages sent to patients increased from 13.82 to 29.83. In multivariable models, weeks with a mix of face-to-face and telemedicine (ß, 0.43; 95% CI, 0.31-0.55; P < .001) visits or entirely telemedicine (ß, 0.91; 95% CI, 0.74-1.09; P < .001) had more EHR time during PSHs than all face-to-face weeks, with similar results for EHR time outside of PSHs. There was no association between telemedicine use and messages received from patients, whereas mixed modalities (ß, -0.90; 95% CI, -1.73 to -0.08; P = .03) and all telemedicine (ß, -4.06; 95% CI, -5.19 to -2.93; P < .001) were associated with fewer messages sent to patients compared with entirely face-to-face weeks. Conclusions and Relevance: The findings of this longitudinal cohort study suggest that telemedicine is associated with greater physician time spent working in the EHR, both during and outside of scheduled hours, mostly documenting visits and not messaging patients. Health systems may need to adjust productivity expectations for physicians and develop strategies to address EHR documentation burden for physicians.

Achieving diagnostic excellence through prevention and teamwork (ADEPT) study protocol: A multicenter, prospective quality and safety program to improve diagnostic processes in medical inpatients.

BACKGROUND: Few hospitals have built surveillance for diagnostic errors into usual care or used comparative quantitative and qualitative data to understand their diagnostic processes and implement interventions designed to reduce these errors. OBJECTIVES: To build surveillance for diagnostic errors into usual care, benchmark diagnostic performance across sites, pilot test interventions, and evaluate the program's impact on diagnostic error rates. METHODS AND ANALYSIS: Achieving diagnostic excellence through prevention and teamwork (ADEPT) is a multicenter, real-world quality and safety program utilizing interrupted time-series techniques to evaluate outcomes. Study subjects will be a randomly sampled population of medical patients hospitalized at 16 US hospitals who died, were transferred to intensive care, or had a rapid response during the hospitalization. Surveillance for diagnostic errors will occur on 10 events per month per site using a previously established two-person adjudication process. Concurrent reviews of patients who had a qualifying event in the previous week will allow for surveys of clinicians to better understand contributors to diagnostic error, or conversely, examples of diagnostic excellence, which cannot be gleaned from medical record review alone. With guidance from national experts in quality and safety, sites will report and benchmark diagnostic error rates, share lessons regarding underlying causes, and design, implement, and pilot test interventions using both Safety I and Safety II approaches aimed at patients, providers, and health systems. Safety II approaches will focus on cases where diagnostic error did not occur, applying theories of how people and systems are able to succeed under varying conditions. The primary outcome will be the number of diagnostic errors per patient, using segmented multivariable regression to evaluate change in y-intercept and change in slope after initiation of the program. ETHICS AND DISSEMINATION: The study has been approved by the University of California, San Francisco Institutional Review Board (IRB), which is serving as the single IRB. Intervention toolkits and study findings will be disseminated through partners including Vizient, The Joint Commission, and Press-Ganey, and through national meetings, scientific journals, and publications aimed at the general public.

Identifying and Addressing Barriers to Implementing Core Electronic Health Record Use Metrics for Ambulatory Care: Virtual Consensus Conference Proceedings.

Precise, reliable, valid metrics that are cost-effective and require reasonable implementation time and effort are needed to drive electronic health record (EHR) improvements and decrease EHR burden. Differences exist between research and vendor definitions of metrics. PROCESS: We convened three stakeholder groups (health system informatics leaders, EHR vendor representatives, and researchers) in a virtual workshop series to achieve consensus on barriers, solutions, and next steps to implementing the core EHR use metrics in ambulatory care. CONCLUSION: Actionable solutions identified to address core categories of EHR metric implementation challenges include: (1) maintaining broad stakeholder engagement, (2) reaching agreement on standardized measure definitions across vendors, (3) integrating clinician perspectives, and (4) addressing cognitive and EHR burden. Building upon the momentum of this workshop's outputs offers promise for overcoming barriers to implementing EHR use metrics.

Social and Medical Care Integration Practices Among Children's Hospitals.

OBJECTIVES: In response to evidence linking social risk factors and adverse health outcomes, new incentives have emerged for hospitals to screen for adverse social determinants of health (SDOH). However, little information is available about the current state of social risk-related care practices among children's hospitals. To address outstanding knowledge gaps, we sought to describe social risk-related care practices among a national sample of children's hospitals. METHODS: We analyzed responses to the 2020 American Hospital Association Annual Survey. Among children's hospitals, we calculated the prevalence of screening for social needs, strategies to address social risks/needs, partnerships with community-based organizations to address social risks/needs at the individual and community level, and rates of impact assessments of how social risk-related interventions affect outcomes. We also used χ2 tests to compare results by hospital characteristics. We weighted results to adjust for nonresponse. RESULTS: The sample included 82 children's hospitals. A total of 79.6% screened for and 96.0% had strategies to address at least 1 social risk factor, although rates varied by SDOH domain. Children's hospitals more commonly partnered with community-based organizations to address patient-level social risks than to participate in community-level initiatives. A total of 39.2% of hospitals assessed SDOH intervention effectiveness. Differences in social risk-related care practices commonly varied by hospital ownership and Medicaid population but not by region. CONCLUSIONS: We found wide variability in social risk-related care practices among children's hospitals based on the risk domain and hospital characteristics. Findings can be used to monitor whether social risk-related care practices change in the setting of new incentives.

Evaluating the comparability of patient-level social risk data extracted from electronic health records: A systematic scoping review.

Objective: To evaluate how and from where social risk data are extracted from EHRs for research purposes, and how observed differences may impact study generalizability. Methods: Systematic scoping review of peer-reviewed literature that used patient-level EHR data to assess 1 ± 6 social risk domains: housing, transportation, food, utilities, safety, social support/isolation. Results: 111/9022 identified articles met inclusion criteria. By domain, social support/isolation was most often included (N = 68/111), predominantly defined by marital/partner status (N = 48/68) and extracted from structured sociodemographic data (N = 45/48). Housing risk was defined primarily by homelessness (N = 39/49). Structured housing data was extracted most from billing codes and screening tools (N = 15/30, 13/30, respectively). Across domains, data were predominantly sourced from structured fields (N = 89/111) versus unstructured free text (N = 32/111). Conclusion: We identified wide variability in how social domains are defined and extracted from EHRs for research. More consistency, particularly in how domains are operationalized, would enable greater insights across studies.

Governance of Electronic Health Record Modification at U.S. Academic Medical Centers.

OBJECTIVES: A key aspect of electronic health record (EHR) governance involves the approach to EHR modification. We report a descriptive study to characterize EHR governance at academic medical centers (AMCs) across the United States. METHODS: We conducted interviews with the Chief Medical Information Officers of 18 AMCs about the process of EHR modification for standard requests. Recordings of the interviews were analyzed to identify categories within prespecified domains. Responses were then assigned to categories for each domain. RESULTS: At our AMCs, EHR requests were governed variably, with a similar number of sites using quantitative scoring systems (7, 38.9%), qualitative systems (5, 27.8%), or no scoring system (6, 33.3%). Two (11%) organizations formally review all requests for their impact on health equity. Although 14 (78%) organizations have trained physician builders/architects, their primary role was not for EHR build. Most commonly reported governance challenges included request volume (11, 61%), integrating diverse clinician input (3, 17%), and stakeholder buy-in (3, 17%). The slowest step in the process was clarifying end user requests (14, 78%). Few leaders had identified metrics for the success of EHR governance. CONCLUSION: Governance approaches for managing EHR modification at AMCs are highly variable, which suggests ongoing efforts to balance EHR standardization and maintenance burden, while dealing with a high volume of requests. Developing metrics to capture the performance of governance and quantify problems may be a key step in identifying best practices.

Lost in Transition: Missed Opportunities for the 4 Ms to Support Post-Acute Care Transitions.

Thousands of health systems have adopted the 4 Ms framework, a set of evidence-based practices specific to older adults, as part of the Age-Friendly Health Systems (AFHS) initiative. However, implementation efforts have largely been setting-specific and approaches to achieve continuity of the 4 Ms during care transitions are nascent. Transitions from hospitals to skilled nursing facilities (SNFs) are one type of care transition that would greatly benefit from continuity of 4 Ms practices. Drawing from the authors' insights and 5 exploratory interviews at 3 health systems that implemented the 4 Ms in the inpatient setting, we describe a set of current-state challenges when trying to extend specific inpatient 4 Ms practices (eg, deprescribing of high-risk medications) as well as the nuanced understanding of the individual's clinical trajectory developed during an inpatient stay. We also offer concrete opportunities, such as developing 4 Ms-centric discharge summary templates, to address the challenges. With the large investment in AFHS transformation and associated efforts to implement the 4 Ms framework in all care settings used by older adults, it is critical to raise awareness of the specific obstacles to promoting continuity of successful 4 Ms practices during care transitions and work to overcome them. Our insights from hospital-to-SNF transitions offer a starting point.

Physician Electronic Health Record Use After Changes in US Centers for Medicare & Medicaid Services Documentation Requirements.

This cohort study examines changes in physician electronic health record (EHR) documentation time before and after changes in Centers for Medicare & Medicaid evaluation and management requirements.