Use of digital patient decision-support tools for atrial fibrillation treatments: a systematic review and meta-analysis.

OBJECTIVES: To assess the effects of digital patient decision-support tools for atrial fibrillation (AF) treatment decisions in adults with AF. STUDY DESIGN: Systematic review and meta-analysis. ELIGIBILITY CRITERIA: Eligible randomised controlled trials (RCTs) evaluated digital patient decision-support tools for AF treatment decisions in adults with AF. INFORMATION SOURCES: We searched MEDLINE, EMBASE and Scopus from 2005 to 2023.Risk-of-bias (RoB) assessment: We assessed RoB using the Cochrane Risk of Bias Tool 2 for RCTs and cluster RCT and the ROBINS-I tool for quasi-experimental studies. SYNTHESIS OF RESULTS: We used random effects meta-analysis to synthesise decisional conflict and patient knowledge outcomes reported in RCTs. We performed narrative synthesis for all outcomes. The main outcomes of interest were decisional conflict and patient knowledge. RESULTS: 13 articles, reporting on 11 studies (4 RCTs, 1 cluster RCT and 6 quasi-experimental) met the inclusion criteria. There were 2714 participants across all studies (2372 in RCTs), of which 26% were women and the mean age was 71 years. Socioeconomically disadvantaged groups were poorly represented in the included studies. Seven studies (n=2508) focused on non-valvular AF and the mean CHAD2DS2-VASc across studies was 3.2 and for HAS-BLED 1.9. All tools focused on decisions regarding thromboembolic stroke prevention and most enabled calculation of individualised stroke risk. Tools were heterogeneous in features and functions; four tools were patient decision aids. The readability of content was reported in one study. Meta-analyses showed a reduction in decisional conflict (4 RCTs (n=2167); standardised mean difference -0.19; 95% CI -0.30 to -0.08; p=0.001; I2=26.5%; moderate certainty evidence) corresponding to a decrease in 12.4 units on a scale of 0 to 100 (95% CI -19.5 to -5.2) and improvement in patient knowledge (2 RCTs (n=1057); risk difference 0.72, 95% CI 0.68, 0.76, p<0.001; I2=0%; low certainty evidence) favouring digital patient decision-support tools compared with usual care. Four of the 11 tools were publicly available and 3 had been implemented in healthcare delivery. CONCLUSIONS: In the context of stroke prevention in AF, digital patient decision-support tools likely reduce decisional conflict and may result in little to no change in patient knowledge, compared with usual care. Future studies should leverage digital capabilities for increased personalisation and interactivity of the tools, with better consideration of health literacy and equity aspects. Additional robust trials and implementation studies are warranted. PROSPERO REGISTRATION NUMBER: CRD42020218025.

Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study.

BACKGROUND: Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. OBJECTIVE: This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. METHODS: A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. RESULTS: The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. CONCLUSIONS: CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

A personalized mobile app for physical activity: An experimental mixed-methods study.

Objectives: To investigate the feasibility of the be.well app and its personalization approach which regularly considers users' preferences, amongst university students. Methods: We conducted a mixed-methods, pre-post experiment, where participants used the app for 2 months. Eligibility criteria included: age 18-34 years; owning an iPhone with Internet access; and fluency in English. Usability was assessed by a validated questionnaire; engagement metrics were reported. Changes in physical activity were assessed by comparing the difference in daily step count between baseline and 2 months. Interviews were conducted to assess acceptability; thematic analysis was conducted. Results: Twenty-three participants were enrolled in the study (mean age = 21.9 years, 71.4% women). The mean usability score was 5.6 ± 0.8 out of 7. The median daily engagement time was 2 minutes. Eighteen out of 23 participants used the app in the last month of the study. Qualitative data revealed that people liked the personalized activity suggestion feature as it was actionable and promoted user autonomy. Some users also expressed privacy concerns if they had to provide a lot of personal data to receive highly personalized features. Daily step count increased after 2 months of the intervention (median difference = 1953 steps/day, p-value <.001, 95% CI 782 to 3112). Conclusions: Incorporating users' preferences in personalized advice provided by a physical activity app was considered feasible and acceptable, with preliminary support for its positive effects on daily step count. Future randomized studies with longer follow up are warranted to determine the effectiveness of personalized mobile apps in promoting physical activity.

Empirical analysis of Zipf's law, power law, and lognormal distributions in medical discharge reports.

BACKGROUND: Bayesian modelling and statistical text analysis rely on informed probability priors to encourage good solutions. OBJECTIVE: This paper empirically analyses whether text in medical discharge reports follow Zipf's law, a commonly assumed statistical property of language where word frequency follows a discrete power-law distribution. METHOD: We examined 20,000 medical discharge reports from the MIMIC-III dataset. Methods included splitting the discharge reports into tokens, counting token frequency, fitting power-law distributions to the data, and testing whether alternative distributions-lognormal, exponential, stretched exponential, and truncated power-law-provided superior fits to the data. RESULT: Discharge reports are best fit by the truncated power-law and lognormal distributions. Discharge reports appear to be near-Zipfian by having the truncated power-law provide superior fits over a pure power-law. CONCLUSION: Our findings suggest that Bayesian modelling and statistical text analysis of discharge report text would benefit from using truncated power-law and lognormal probability priors and non-parametric models that capture power-law behavior.

Identifying relevant information in medical conversations to summarize a clinician-patient encounter.

To inform the development of automated summarization of clinical conversations, this study sought to estimate the proportion of doctor-patient communication in general practice (GP) consultations used for generating a consultation summary. Two researchers with a medical degree read the transcripts of 44 GP consultations and highlighted the phrases to be used for generating a summary of the consultation. For all consultations, less than 20% of all words in the transcripts were needed for inclusion in the summary. On average, 9.1% of all words in the transcripts, 26.6% of all medical terms, and 27.3% of all speaker turns were highlighted. The results indicate that communication content used for generating a consultation summary makes up a small portion of GP consultations, and automated summarization solutions-such as digital scribes-must focus on identifying the 20% relevant information for automatically generating consultation summaries.

An Electronic Clinical Decision Support System for the Management of Low Back Pain in Community Pharmacy: Development and Mixed Methods Feasibility Study.

BACKGROUND: People with low back pain (LBP) in the community often do not receive evidence-based advice and management. Community pharmacists can play an important role in supporting people with LBP as pharmacists are easily accessible to provide first-line care. However, previous research suggests that pharmacists may not consistently deliver advice that is concordant with guideline recommendations and may demonstrate difficulty determining which patients require prompt medical review. A clinical decision support system (CDSS) may enhance first-line care of LBP, but none exists to support the community pharmacist-client consultation. OBJECTIVE: This study aimed to develop a CDSS to guide first-line care of LBP in the community pharmacy setting and to evaluate the pharmacist-reported usability and acceptance of the prototype system. METHODS: A cross-platform Web app for the Apple iPad was developed in conjunction with academic and clinical experts using an iterative user-centered design process during interface design, clinical reasoning, program development, and evaluation. The CDSS was evaluated via one-to-one user-testing with 5 community pharmacists (5 case vignettes each). Data were collected via video recording, screen capture, survey instrument (system usability scale), and direct observation. RESULTS: Pharmacists' agreement with CDSS-generated self-care recommendations was 90% (18/20), with medicines recommendations was 100% (25/25), and with referral advice was 88% (22/25; total 70 recommendations). Pharmacists expressed uncertainty when screening for serious pathology in 40% (10/25) of cases. Pharmacists requested more direction from the CDSS in relation to automated prompts for user input and page navigation. Overall system usability was rated as excellent (mean score 92/100, SD 6.5; 90th percentile compared with similar systems), with acceptance rated as good to excellent. CONCLUSIONS: A novel CDSS (high-fidelity prototype) to enhance pharmacist care of LBP was developed, underpinned by clinical practice guidelines and informed by a multidisciplinary team of experts. User-testing revealed a high level of usability and acceptance of the prototype system, with suggestions to improve interface prompts and information delivery. The small study sample limits the generalizability of the findings but offers important insights to inform the next stage of system development.

Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures.

BACKGROUND: Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks often remain understudied. OBJECTIVE: This study aimed to analyze how commonly available, general-purpose CAs on smartphones and smart speakers respond to health and lifestyle prompts (questions and open-ended statements) by examining their responses in terms of content and structure alike. METHODS: We followed a piloted script to present health- and lifestyle-related prompts to 8 CAs. The CAs' responses were assessed for their appropriateness on the basis of the prompt type: responses to safety-critical prompts were deemed appropriate if they included a referral to a health professional or service, whereas responses to lifestyle prompts were deemed appropriate if they provided relevant information to address the problem prompted. The response structure was also examined according to information sources (Web search-based or precoded), response content style (informative and/or directive), confirmation of prompt recognition, and empathy. RESULTS: The 8 studied CAs provided in total 240 responses to 30 prompts. They collectively responded appropriately to 41% (46/112) of the safety-critical and 39% (37/96) of the lifestyle prompts. The ratio of appropriate responses deteriorated when safety-critical prompts were rephrased or when the agent used a voice-only interface. The appropriate responses included mostly directive content and empathy statements for the safety-critical prompts and a mix of informative and directive content for the lifestyle prompts. CONCLUSIONS: Our results suggest that the commonly available, general-purpose CAs on smartphones and smart speakers with unconstrained natural language interfaces are limited in their ability to advise on both the safety-critical health prompts and lifestyle prompts. Our study also identified some response structures the CAs employed to present their appropriate responses. Further investigation is needed to establish guidelines for designing suitable response structures for different prompt types.

Challenges of developing a digital scribe to reduce clinical documentation burden.

Clinicians spend a large amount of time on clinical documentation of patient encounters, often impacting quality of care and clinician satisfaction, and causing physician burnout. Advances in artificial intelligence (AI) and machine learning (ML) open the possibility of automating clinical documentation with digital scribes, using speech recognition to eliminate manual documentation by clinicians or medical scribes. However, developing a digital scribe is fraught with problems due to the complex nature of clinical environments and clinical conversations. This paper identifies and discusses major challenges associated with developing automated speech-based documentation in clinical settings: recording high-quality audio, converting audio to transcripts using speech recognition, inducing topic structure from conversation data, extracting medical concepts, generating clinically meaningful summaries of conversations, and obtaining clinical data for AI and ML algorithms.

The Personalization of Conversational Agents in Health Care: Systematic Review.

BACKGROUND: The personalization of conversational agents with natural language user interfaces is seeing increasing use in health care applications, shaping the content, structure, or purpose of the dialogue between humans and conversational agents. OBJECTIVE: The goal of this systematic review was to understand the ways in which personalization has been used with conversational agents in health care and characterize the methods of its implementation. METHODS: We searched on PubMed, Embase, CINAHL, PsycInfo, and ACM Digital Library using a predefined search strategy. The studies were included if they: (1) were primary research studies that focused on consumers, caregivers, or health care professionals; (2) involved a conversational agent with an unconstrained natural language interface; (3) tested the system with human subjects; and (4) implemented personalization features. RESULTS: The search found 1958 publications. After abstract and full-text screening, 13 studies were included in the review. Common examples of personalized content included feedback, daily health reports, alerts, warnings, and recommendations. The personalization features were implemented without a theoretical framework of customization and with limited evaluation of its impact. While conversational agents with personalization features were reported to improve user satisfaction, user engagement and dialogue quality, the role of personalization in improving health outcomes was not assessed directly. CONCLUSIONS: Most of the studies in our review implemented the personalization features without theoretical or evidence-based support for them and did not leverage the recent developments in other domains of personalization. Future research could incorporate personalization as a distinct design factor with a more careful consideration of its impact on health outcomes and its implications on patient safety, privacy, and decision-making.

A network model of activities in primary care consultations.

OBJECTIVE: The objective of this study is to characterize the dynamic structure of primary care consultations by identifying typical activities and their inter-relationships to inform the design of automated approaches to clinical documentation using natural language processing and summarization methods. MATERIALS AND METHODS: This is an observational study in Australian general practice involving 31 consultations with 4 primary care physicians. Consultations were audio-recorded, and computer interactions were recorded using screen capture. Physical interactions in consultation rooms were noted by observers. Brief interviews were conducted after consultations. Conversational transcripts were analyzed to identify different activities and their speech content as well as verbal cues signaling activity transitions. An activity transition analysis was then undertaken to generate a network of activities and transitions. RESULTS: Observed activity classes followed those described in well-known primary care consultation models. Activities were often fragmented across consultations, did not flow necessarily in a defined order, and the flow between activities was nonlinear. Modeling activities as a network revealed that discussing a patient's present complaint was the most central activity and was highly connected to medical history taking, physical examination, and assessment, forming a highly interrelated bundle. Family history, allergy, and investigation discussions were less connected suggesting less dependency on other activities. Clear verbal signs were often identifiable at transitions between activities. DISCUSSION: Primary care consultations do not appear to follow a classic linear model of defined information seeking activities; rather, they are fragmented, highly interdependent, and can be reactively triggered. CONCLUSION: The nonlinearity of activities has significant implications for the design of automated information capture. Whereas dictation systems generate literal translation of speech into text, speech-based clinical summary systems will need to link disparate information fragments, merge their content, and abstract coherent information summaries.

Conversational agents in healthcare: a systematic review.

Objective: Our objective was to review the characteristics, current applications, and evaluation measures of conversational agents with unconstrained natural language input capabilities used for health-related purposes. Methods: We searched PubMed, Embase, CINAHL, PsycInfo, and ACM Digital using a predefined search strategy. Studies were included if they focused on consumers or healthcare professionals; involved a conversational agent using any unconstrained natural language input; and reported evaluation measures resulting from user interaction with the system. Studies were screened by independent reviewers and Cohen's kappa measured inter-coder agreement. Results: The database search retrieved 1513 citations; 17 articles (14 different conversational agents) met the inclusion criteria. Dialogue management strategies were mostly finite-state and frame-based (6 and 7 conversational agents, respectively); agent-based strategies were present in one type of system. Two studies were randomized controlled trials (RCTs), 1 was cross-sectional, and the remaining were quasi-experimental. Half of the conversational agents supported consumers with health tasks such as self-care. The only RCT evaluating the efficacy of a conversational agent found a significant effect in reducing depression symptoms (effect size d = 0.44, p = .04). Patient safety was rarely evaluated in the included studies. Conclusions: The use of conversational agents with unconstrained natural language input capabilities for health-related purposes is an emerging field of research, where the few published studies were mainly quasi-experimental, and rarely evaluated efficacy or safety. Future studies would benefit from more robust experimental designs and standardized reporting. Protocol Registration: The protocol for this systematic review is registered at PROSPERO with the number CRD42017065917.