Generative artificial intelligence responses to patient messages in the electronic health record: early lessons learned.

Background: Electronic health record (EHR)-based patient messages can contribute to burnout. Messages with a negative tone are particularly challenging to address. In this perspective, we describe our initial evaluation of large language model (LLM)-generated responses to negative EHR patient messages and contend that using LLMs to generate initial drafts may be feasible, although refinement will be needed. Methods: A retrospective sample (n = 50) of negative patient messages was extracted from a health system EHR, de-identified, and inputted into an LLM (ChatGPT). Qualitative analyses were conducted to compare LLM responses to actual care team responses. Results: Some LLM-generated draft responses varied from human responses in relational connection, informational content, and recommendations for next steps. Occasionally, the LLM draft responses could have potentially escalated emotionally charged conversations. Conclusion: Further work is needed to optimize the use of LLMs for responding to negative patient messages in the EHR.

Sustainable Approach to Justice, Equity, Diversity, and Inclusion Through Better Quality Measurement.

The US health care industry has broadly adopted performance and quality measures that are extracted from electronic health records and connected to payment incentives that hope to improve declining life expectancy and health status and reduce costs. While the development of a quality measurement infrastructure based on electronic health record data was an important first step in addressing US health outcomes, these metrics, reflecting the average performance across diverse populations, do not adequately adjust for population demographic differences, social determinants of health, or ecosystem vulnerability. Like society as a whole, health care must confront the powerful impact that social determinants of health, race, ethnicity, and other demographic variations have on key health care performance indicators and quality metrics. Tools that are currently available to capture and report the health status of Americans lack the granularity, complexity, and standardization needed to improve health and address disparities at the local level. In this article, we discuss the current and future state of electronic clinical quality measures through a lens of equity.

Distribution of COVID-19 Home Testing Through Community Health Centers: Results of the COVID CoNOce MÁS Study.

OBJECTIVE: To assess acceptability and feasibility of rapid at-home COVID-19 testing and reporting of test results among individuals seeking care at community health centers (CHCs) and their household members. METHODS: Participants were recruited from 2 Community Health Centers during a clinic visit or a community event. Over-the-counter COVID-19 tests were distributed to participants for self-testing and to offer testing to household members. Separate surveys were administered to collect baseline information on the study participant and to collect test results on the study participant and household members. We calculated the proportion of individuals who agreed to complete COVID home testing, those who reported test results, and the test positivity. For household members, we calculated the proportion who completed and reported results and the positivity rate. We assessed reasons for undergoing COVID-19 testing and the action taken by participants who reported positive tests. RESULTS: A total of 2189 individuals were approached by CHC staff for participation and 1013 (46.3%) agreed to participate. Among the 959 participants with complete sociodemographic data, 88% were Hispanic and 82.6% were female. The proportion providing test results was 36.2% and the test positivity was 4.2%. Among the 1927 test reports, 35.3% for the index participant and 64.4% were for household members. The largest proportion of test results were for index participants (35.3%) and the second largest was for the participant's children (32.1%), followed by parents (16.9%), and spouse/partner (13.2%). The 2 most common reasons for testing were symptoms (29%) and attending family gatherings (26%). Among test-positive individuals (n = 80), most (83.3%) noted that they isolated but only 16.3% called their provider and 1.3% visited a clinic. CONCLUSION: Our results show interest in at-home COVID-19 testing of multiple household members, as we headed into the endemic phase of the pandemic. However, reporting of test results was modest and among test-positive individuals, reporting results to a provider was very low. These results underscore the challenges with reporting and following guidelines among people undergoing home testing for COVID-19, which may have implications for future pandemics.

AI-Generated Draft Replies Integrated Into Health Records and Physicians' Electronic Communication.

Importance: Timely tests are warranted to assess the association between generative artificial intelligence (GenAI) use and physicians' work efforts. Objective: To investigate the association between GenAI-drafted replies for patient messages and physician time spent on answering messages and the length of replies. Design, Setting, and Participants: Randomized waiting list quality improvement (QI) study from June to August 2023 in an academic health system. Primary care physicians were randomized to an immediate activation group and a delayed activation group. Data were analyzed from August to November 2023. Exposure: Access to GenAI-drafted replies for patient messages. Main Outcomes and Measures: Time spent (1) reading messages, (2) replying to messages, (3) length of replies, and (4) physician likelihood to recommend GenAI drafts. The a priori hypothesis was that GenAI drafts would be associated with less physician time spent reading and replying to messages. A mixed-effects model was used. Results: Fifty-two physicians participated in this QI study, with 25 randomized to the immediate activation group and 27 randomized to the delayed activation group. A contemporary control group included 70 physicians. There were 18 female participants (72.0%) in the immediate group and 17 female participants (63.0%) in the delayed group; the median age range was 35-44 years in the immediate group and 45-54 years in the delayed group. The median (IQR) time spent reading messages in the immediate group was 26 (11-69) seconds at baseline, 31 (15-70) seconds 3 weeks after entry to the intervention, and 31 (14-70) seconds 6 weeks after entry. The delayed group's median (IQR) read time was 25 (10-67) seconds at baseline, 29 (11-77) seconds during the 3-week waiting period, and 32 (15-72) seconds 3 weeks after entry to the intervention. The contemporary control group's median (IQR) read times were 21 (9-54), 22 (9-63), and 23 (9-60) seconds in corresponding periods. The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008), a -5.9% change in reply time (95% CI, -16.6% to 6.2%; P = .33), and a 17.9% increase in reply length (95% CI, 10.1% to 26.2%; P < .001). Participants recognized GenAI's value and suggested areas for improvement. Conclusions and Relevance: In this QI study, GenAI-drafted replies were associated with significantly increased read time, no change in reply time, significantly increased reply length, and some perceived benefits. Rigorous empirical tests are necessary to further examine GenAI's performance. Future studies should examine patient experience and compare multiple GenAIs, including those with medical training.