Large Language Model-Based Responses to Patients' In-Basket Messages.

Importance: Virtual patient-physician communications have increased since 2020 and negatively impacted primary care physician (PCP) well-being. Generative artificial intelligence (GenAI) drafts of patient messages could potentially reduce health care professional (HCP) workload and improve communication quality, but only if the drafts are considered useful. Objectives: To assess PCPs' perceptions of GenAI drafts and to examine linguistic characteristics associated with equity and perceived empathy. Design, Setting, and Participants: This cross-sectional quality improvement study tested the hypothesis that PCPs' ratings of GenAI drafts (created using the electronic health record [EHR] standard prompts) would be equivalent to HCP-generated responses on 3 dimensions. The study was conducted at NYU Langone Health using private patient-HCP communications at 3 internal medicine practices piloting GenAI. Exposures: Randomly assigned patient messages coupled with either an HCP message or the draft GenAI response. Main Outcomes and Measures: PCPs rated responses' information content quality (eg, relevance), using a Likert scale, communication quality (eg, verbosity), using a Likert scale, and whether they would use the draft or start anew (usable vs unusable). Branching logic further probed for empathy, personalization, and professionalism of responses. Computational linguistics methods assessed content differences in HCP vs GenAI responses, focusing on equity and empathy. Results: A total of 16 PCPs (8 [50.0%] female) reviewed 344 messages (175 GenAI drafted; 169 HCP drafted). Both GenAI and HCP responses were rated favorably. GenAI responses were rated higher for communication style than HCP responses (mean [SD], 3.70 [1.15] vs 3.38 [1.20]; P = .01, U = 12 568.5) but were similar to HCPs on information content (mean [SD], 3.53 [1.26] vs 3.41 [1.27]; P = .37; U = 13 981.0) and usable draft proportion (mean [SD], 0.69 [0.48] vs 0.65 [0.47], P = .49, t = -0.6842). Usable GenAI responses were considered more empathetic than usable HCP responses (32 of 86 [37.2%] vs 13 of 79 [16.5%]; difference, 125.5%), possibly attributable to more subjective (mean [SD], 0.54 [0.16] vs 0.31 [0.23]; P < .001; difference, 74.2%) and positive (mean [SD] polarity, 0.21 [0.14] vs 0.13 [0.25]; P = .02; difference, 61.5%) language; they were also numerically longer (mean [SD] word count, 90.5 [32.0] vs 65.4 [62.6]; difference, 38.4%), but the difference was not statistically significant (P = .07) and more linguistically complex (mean [SD] score, 125.2 [47.8] vs 95.4 [58.8]; P = .002; difference, 31.2%). Conclusions: In this cross-sectional study of PCP perceptions of an EHR-integrated GenAI chatbot, GenAI was found to communicate information better and with more empathy than HCPs, highlighting its potential to enhance patient-HCP communication. However, GenAI drafts were less readable than HCPs', a significant concern for patients with low health or English literacy.

Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work.

The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians' after-hours clinical work ("work-outside-work"). The surge in patients' digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians' after-hours commitments. We examined the impact on physicians' workload from two types of digital demands - patients' messages requesting medical advice (PMARs) sent to physicians' inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect of PMARs on physicians' work-outside-work and that this relationship is moderated by physicians' specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewer PMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increased PMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. Reducing PMAR volumes and efficient inbasket management strategies needed to reduce physicians' work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.

Evidence for Telemedicine's Ongoing Transformation of Health Care Delivery Since the Onset of COVID-19: Retrospective Observational Study.

BACKGROUND: The surge of telemedicine use during the early stages of the COVID-19 pandemic has been well documented. However, scarce evidence considers the use of telemedicine in the subsequent period. OBJECTIVE: This study aims to evaluate use patterns of video-based telemedicine visits for ambulatory care and urgent care provision over the course of recurring pandemic waves in 1 large health system in New York City (NYC) and what this means for health care delivery. METHODS: Retrospective electronic health record (EHR) data of patients from January 1, 2020, to February 28, 2022, were used to longitudinally track and analyze telemedicine and in-person visit volumes across ambulatory care specialties and urgent care, as well as compare them to a prepandemic baseline (June-November 2019). Diagnosis codes to differentiate suspected COVID-19 visits from non-COVID-19 visits, as well as evaluating COVID-19-based telemedicine use over time, were compared to the total number of COVID-19-positive cases in the same geographic region (city level). The time series data were segmented based on change-point analysis, and variances in visit trends were compared between the segments. RESULTS: The emergence of COVID-19 prompted an early increase in the number of telemedicine visits across the urgent care and ambulatory care settings. This use continued throughout the pandemic at a much higher level than the prepandemic baseline for both COVID-19 and non-COVID-19 suspected visits, despite the fluctuation in COVID-19 cases throughout the pandemic and the resumption of in-person clinical services. The use of telemedicine-based urgent care services for COVID-19 suspected visits showed more variance in response to each pandemic wave, but telemedicine visits for ambulatory care have remained relatively steady after the initial crisis period. During the Omicron wave, the use of all visit types, including in-person activities, decreased. Patients between 25 and 34 years of age were the largest users of telemedicine-based urgent care. Patient satisfaction with telemedicine-based urgent care remained high despite the rapid scaling of services to meet increased demand. CONCLUSIONS: The trend of the increased use of telemedicine as a means of health care delivery relative to the pre-COVID-19 baseline has been maintained throughout the later pandemic periods despite fluctuating COVID-19 cases and the resumption of in-person care delivery. Overall satisfaction with telemedicine-based care is also high. The trends in telemedicine use suggest that telemedicine-based health care delivery has become a mainstream and sustained supplement to in-person-based ambulatory care, particularly for younger patients, for both urgent and nonurgent care needs. These findings have implications for the health care delivery system, including practice leaders, insurers, and policymakers. Further investigation is needed to evaluate telemedicine adoption by key demographics, identify ongoing barriers to adoption, and explore the impacts of sustained use of telemedicine on health care outcomes and experience.

The Impact of Telemedicine on Physicians' After-hours Electronic Health Record "Work Outside Work" During the COVID-19 Pandemic: Retrospective Cohort Study.

BACKGROUND: Telemedicine as a mode of health care work has grown dramatically during the COVID-19 pandemic; the impact of this transition on clinicians' after-hours electronic health record (EHR)-based clinical and administrative work is unclear. OBJECTIVE: This study assesses the impact of the transition to telemedicine during the COVID-19 pandemic on physicians' EHR-based after-hours workload (ie, "work outside work") at a large academic medical center in New York City. METHODS: We conducted an EHR-based retrospective cohort study of ambulatory care physicians providing telemedicine services before the pandemic, during the acute pandemic, and after the acute pandemic, relating EHR-based after-hours work to telemedicine intensity (ie, percentage of care provided via telemedicine) and clinical load (ie, patient load per provider). RESULTS: A total of 2129 physicians were included in this study. During the acute pandemic, the volume of care provided via telemedicine significantly increased for all physicians, whereas patient volume decreased. When normalized by clinical load (ie, average appointments per day by average clinical days per week), telemedicine intensity was positively associated with work outside work across time periods. This association was strongest after the acute pandemic. CONCLUSIONS: Taking physicians' clinical load into account, physicians who devoted a higher proportion of their clinical time to telemedicine throughout various stages of the pandemic engaged in higher levels of EHR-based after-hours work compared to those who used telemedicine less intensively. This suggests that telemedicine, as currently delivered, may be less efficient than in-person-based care and may increase the after-hours work burden of physicians.

Quantitative and Qualitative Evaluation of Provider Use of a Novel Machine Learning Model for Favorable Outcome Prediction.

Predictive models may be particularly beneficial to clinicians when they face uncertainty and seek to develop a mental model of disease progression, but we know little about the post-implementation effects of predictive models on clinicians' experience of their work. Combining survey and interview methods, we found that providers using a predictive algorithm reported being significantly less uncertain and better able to anticipate, plan and prepare for patient discharge than non-users. The tool helped hospitalists form and develop confidence in their mental models of a novel disease (Covid-19). Yet providers' attention to the predictive tool declined as their confidence in their own mental models grew. Predictive algorithms that not only offer data but also provide feedback on decisions, thus supporting providers' motivation for continuous learning, hold promise for more sustained provider attention and cognition augmentation.

Preferences and patterns of response to public health advice during the COVID-19 pandemic.

With recurring waves of the Covid-19 pandemic, a dilemma facing public health leadership is whether to provide public advice that is medically optimal (e.g., most protective against infection if followed), but unlikely to be adhered to, or advice that is less protective but is more likely to be followed. To provide insight about this dilemma, we examined and quantified public perceptions about the tradeoff between (a) the stand-alone value of health behavior advice, and (b) the advice's adherence likelihood. In a series of studies about preference for public health leadership advice, we asked 1061 participants to choose between (5) strict advice that is medically optimal if adhered to but which is less likely to be broadly followed, and (2) relaxed advice, which is less medically effective but more likely to gain adherence-given varying infection expectancies. Participants' preference was consistent with risk aversion. Offering an informed choice alternative that shifts volition to advice recipients only strengthened risk aversion, but also demonstrated that informed choice was preferred as much or more than the risk-averse strict advice.

Who Represents Our Group? The Effects of Prototype Content on Perceived Status Dispersion and Social Undermining.

Group identity may be embodied in more typical or extreme member attributes. The present research suggests that individuals' perceptions of the group identity prototype predict their beliefs about the status hierarchy and, in turn, the prevalence of social undermining behavior. Across four studies using both experimental and field data, we find that perceiving that the group prototype is focused on the ideal rather than the central tendency is associated with greater levels of perceived status dispersion and social undermining, and that perceived status dispersion mediates the relationship between members' perception of the group prototype and social undermining behavior. We also find that social context-specifically, salient group achievement goals elicited by intergroup competition and common ingroup identity-attenuates the effect of ideal prototypes on perceived social undermining. Theoretical implications for the social identity, status, and social undermining literatures are discussed.