Potential of Large Language Models in Health Care: Delphi Study.

BACKGROUND: A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications. OBJECTIVE: The aim of this adapted Delphi study was to collect researchers' opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care. METHODS: We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items. RESULTS: The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data. CONCLUSIONS: Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.

Longitudinal Coadaptation of Older Adults With Wearables and Voice-Activated Virtual Assistants: Scoping Review.

BACKGROUND: The integration of smart technologies, including wearables and voice-activated devices, is increasingly recognized for enhancing the independence and well-being of older adults. However, the long-term dynamics of their use and the coadaptation process with older adults remain poorly understood. This scoping review explores how interactions between older adults and smart technologies evolve over time to improve both user experience and technology utility. OBJECTIVE: This review synthesizes existing research on the coadaptation between older adults and smart technologies, focusing on longitudinal changes in use patterns, the effectiveness of technological adaptations, and the implications for future technology development and deployment to improve user experiences. METHODS: Following the Joanna Briggs Institute Reviewer's Manual and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, this scoping review examined peer-reviewed papers from databases including Ovid MEDLINE, Ovid Embase, PEDro, Ovid PsycINFO, and EBSCO CINAHL from the year 2000 to August 28, 2023, and included forward and backward searches. The search was updated on March 1, 2024. Empirical studies were included if they involved (1) individuals aged 55 years or older living independently and (2) focused on interactions and adaptations between older adults and wearables and voice-activated virtual assistants in interventions for a minimum period of 8 weeks. Data extraction was informed by the selection and optimization with compensation framework and the sex- and gender-based analysis plus theoretical framework and used a directed content analysis approach. RESULTS: The search yielded 16,143 papers. Following title and abstract screening and a full-text review, 5 papers met the inclusion criteria. Study populations were mostly female participants and aged 73-83 years from the United States and engaged with voice-activated virtual assistants accessed through smart speakers and wearables. Users frequently used simple commands related to music and weather, integrating devices into daily routines. However, communication barriers often led to frustration due to devices' inability to recognize cues or provide personalized responses. The findings suggest that while older adults can integrate smart technologies into their lives, a lack of customization and user-friendly interfaces hinder long-term adoption and satisfaction. The studies highlight the need for technology to be further developed so they can better meet this demographic's evolving needs and call for research addressing small sample sizes and limited diversity. CONCLUSIONS: Our findings highlight a critical need for continued research into the dynamic and reciprocal relationship between smart technologies and older adults over time. Future studies should focus on more diverse populations and extend monitoring periods to provide deeper insights into the coadaptation process. Insights gained from this review are vital for informing the development of more intuitive, user-centric smart technology solutions to better support the aging population in maintaining independence and enhancing their quality of life. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/51129.

Transformer Models in Healthcare: A Survey and Thematic Analysis of Potentials, Shortcomings and Risks.

Large Language Models (LLMs) such as General Pretrained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT), which use transformer model architectures, have significantly advanced artificial intelligence and natural language processing. Recognized for their ability to capture associative relationships between words based on shared context, these models are poised to transform healthcare by improving diagnostic accuracy, tailoring treatment plans, and predicting patient outcomes. However, there are multiple risks and potentially unintended consequences associated with their use in healthcare applications. This study, conducted with 28 participants using a qualitative approach, explores the benefits, shortcomings, and risks of using transformer models in healthcare. It analyses responses to seven open-ended questions using a simplified thematic analysis. Our research reveals seven benefits, including improved operational efficiency, optimized processes and refined clinical documentation. Despite these benefits, there are significant concerns about the introduction of bias, auditability issues and privacy risks. Challenges include the need for specialized expertise, the emergence of ethical dilemmas and the potential reduction in the human element of patient care. For the medical profession, risks include the impact on employment, changes in the patient-doctor dynamic, and the need for extensive training in both system operation and data interpretation.

Sentiment analysis of clinical narratives: A scoping review.

A clinical sentiment is a judgment, thought or attitude promoted by an observation with respect to the health of an individual. Sentiment analysis has drawn attention in the healthcare domain for secondary use of data from clinical narratives, with a variety of applications including predicting the likelihood of emerging mental illnesses or clinical outcomes. The current state of research has not yet been summarized. This study presents results from a scoping review aiming at providing an overview of sentiment analysis of clinical narratives in order to summarize existing research and identify open research gaps. The scoping review was carried out in line with the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) guideline. Studies were identified by searching 4 electronic databases (e.g., PubMed, IEEE Xplore) in addition to conducting backward and forward reference list checking of the included studies. We extracted information on use cases, methods and tools applied, used datasets and performance of the sentiment analysis approach. Of 1,200 citations retrieved, 29 unique studies were included in the review covering a period of 8 years. Most studies apply general domain tools (e.g. TextBlob) and sentiment lexicons (e.g. SentiWordNet) for realizing use cases such as prediction of clinical outcomes; others proposed new domain-specific sentiment analysis approaches based on machine learning. Accuracy values between 71.5-88.2% are reported. Data used for evaluation and test are often retrieved from MIMIC databases or i2b2 challenges. Latest developments related to artificial neural networks are not yet fully considered in this domain. We conclude that future research should focus on developing a gold standard sentiment lexicon, adapted to the specific characteristics of clinical narratives. Efforts have to be made to either augment existing or create new high-quality labeled data sets of clinical narratives. Last, the suitability of state-of-the-art machine learning methods for natural language processing and in particular transformer-based models should be investigated for their application for sentiment analysis of clinical narratives.

Designing personalised mHealth solutions: An overview.

INTRODUCTION: Mobile health, or mHealth, is based on mobile information and communication technologies and provides solutions for empowering individuals to participate in healthcare. Personalisation techniques have been used to increase user engagement and adherence to interventions delivered as mHealth solutions. This study aims to explore the current state of personalisation in mHealth, including its current trends and implementation. MATERIALS AND METHODS: We conducted a review following PRISMA guidelines. Four databases (PubMed, ACM Digital Library, IEEE Xplore, and APA PsycInfo) were searched for studies on mHealth solutions that integrate personalisation. The retrieved papers were assessed for eligibility and useful information regarding integrated personalisation techniques. RESULTS: Out of the 1,139 retrieved studies, 62 were included in the narrative synthesis. Research interest in the personalisation of mHealth solutions has increased since 2020. mHealth solutions were mainly applied to endocrine, nutritional, and metabolic diseases; mental, behavioural, or neurodevelopmental diseases; or the promotion of healthy lifestyle behaviours. Its main purposes are to support disease self-management and promote healthy lifestyle behaviours. Mobile applications are the most prevalent technological solution. Although several design models, such as user-centred and patient-centred designs, were used, no specific frameworks or models for personalisation were followed. These solutions rely on behaviour change theories, use gamification or motivational messages, and personalise the content rather than functionality. A broad range of data is used for personalisation purposes. There is a lack of studies assessing the efficacy of these solutions; therefore, further evidence is needed. DISCUSSION: Personalisation in mHealth has not been well researched. Although several techniques have been integrated, the effects of using a combination of personalisation techniques remain unclear. Although personalisation is considered a persuasive strategy, many mHealth solutions do not employ it. CONCLUSIONS: Open research questions concern guidelines for successful personalisation techniques in mHealth, design frameworks, and comprehensive studies on the effects and interactions among multiple personalisation techniques.

Developing a Technical-Oriented Taxonomy to Define Archetypes of Conversational Agents in Health Care: Literature Review and Cluster Analysis.

BACKGROUND: The evolution of artificial intelligence and natural language processing generates new opportunities for conversational agents (CAs) that communicate and interact with individuals. In the health domain, CAs became popular as they allow for simulating the real-life experience in a health care setting, which is the conversation with a physician. However, it is still unclear which technical archetypes of health CAs can be distinguished. Such technical archetypes are required, among other things, for harmonizing evaluation metrics or describing the landscape of health CAs. OBJECTIVE: The objective of this work was to develop a technical-oriented taxonomy for health CAs and characterize archetypes of health CAs based on their technical characteristics. METHODS: We developed a taxonomy of technical characteristics for health CAs based on scientific literature and empirical data and by applying a taxonomy development framework. To demonstrate the applicability of the taxonomy, we analyzed the landscape of health CAs of the last years based on a literature review. To form technical design archetypes of health CAs, we applied a k-means clustering method. RESULTS: Our taxonomy comprises 18 unique dimensions corresponding to 4 perspectives of technical characteristics (setting, data processing, interaction, and agent appearance). Each dimension consists of 2 to 5 characteristics. The taxonomy was validated based on 173 unique health CAs that were identified out of 1671 initially retrieved publications. The 173 CAs were clustered into 4 distinctive archetypes: a text-based ad hoc supporter; a multilingual, hybrid ad hoc supporter; a hybrid, single-language temporary advisor; and, finally, an embodied temporary advisor, rule based with hybrid input and output options. CONCLUSIONS: From the cluster analysis, we learned that the time dimension is important from a technical perspective to distinguish health CA archetypes. Moreover, we were able to identify additional distinctive, dominant characteristics that are relevant when evaluating health-related CAs (eg, input and output options or the complexity of the CA personality). Our archetypes reflect the current landscape of health CAs, which is characterized by rule based, simple systems in terms of CA personality and interaction. With an increase in research interest in this field, we expect that more complex systems will arise. The archetype-building process should be repeated after some time to check whether new design archetypes emerge.

Correction: Wearable Artificial Intelligence for Anxiety and Depression: Scoping Review.

[This corrects the article DOI: 10.2196/42672.].

Wearable Artificial Intelligence for Anxiety and Depression: Scoping Review.

BACKGROUND: Anxiety and depression are the most common mental disorders worldwide. Owing to the lack of psychiatrists around the world, the incorporation of artificial intelligence (AI) into wearable devices (wearable AI) has been exploited to provide mental health services. OBJECTIVE: This review aimed to explore the features of wearable AI used for anxiety and depression to identify application areas and open research issues. METHODS: We searched 8 electronic databases (MEDLINE, PsycINFO, Embase, CINAHL, IEEE Xplore, ACM Digital Library, Scopus, and Google Scholar) and included studies that met the inclusion criteria. Then, we checked the studies that cited the included studies and screened studies that were cited by the included studies. The study selection and data extraction were carried out by 2 reviewers independently. The extracted data were aggregated and summarized using narrative synthesis. RESULTS: Of the 1203 studies identified, 69 (5.74%) were included in this review. Approximately, two-thirds of the studies used wearable AI for depression, whereas the remaining studies used it for anxiety. The most frequent application of wearable AI was in diagnosing anxiety and depression; however, none of the studies used it for treatment purposes. Most studies targeted individuals aged between 18 and 65 years. The most common wearable device used in the studies was Actiwatch AW4 (Cambridge Neurotechnology Ltd). Wrist-worn devices were the most common type of wearable device in the studies. The most commonly used category of data for model development was physical activity data, followed by sleep data and heart rate data. The most frequently used data set from open sources was Depresjon. The most commonly used algorithm was random forest, followed by support vector machine. CONCLUSIONS: Wearable AI can offer great promise in providing mental health services related to anxiety and depression. Wearable AI can be used by individuals for the prescreening assessment of anxiety and depression. Further reviews are needed to statistically synthesize the studies' results related to the performance and effectiveness of wearable AI. Given its potential, technology companies should invest more in wearable AI for the treatment of anxiety and depression.

Implementation of Cognitive Behavioral Therapy in e-Mental Health Apps: Literature Review.

BACKGROUND: To address the matter of limited resources for treating individuals with mental disorders, e-mental health has gained interest in recent years. More specifically, mobile health (mHealth) apps have been suggested as electronic mental health interventions accompanied by cognitive behavioral therapy (CBT). OBJECTIVE: This study aims to identify the therapeutic aspects of CBT that have been implemented in existing mHealth apps and the technologies used. From these, we aim to derive research gaps that should be addressed in the future. METHODS: Three databases were screened for studies on mHealth apps in the context of mental disorders that implement techniques of CBT: PubMed, IEEE Xplore, and ACM Digital Library. The studies were independently selected by 2 reviewers, who then extracted data from the included studies. Data on CBT techniques and their technical implementation in mHealth apps were synthesized narratively. RESULTS: Of the 530 retrieved citations, 34 (6.4%) studies were included in this review. mHealth apps for CBT exploit two groups of technologies: technologies that implement CBT techniques for cognitive restructuring, behavioral activation, and problem solving (exposure is not yet realized in mHealth apps) and technologies that aim to increase user experience, adherence, and engagement. The synergy of these technologies enables patients to self-manage and self-monitor their mental state and access relevant information on their mental illness, which helps them cope with mental health problems and allows self-treatment. CONCLUSIONS: There are CBT techniques that can be implemented in mHealth apps. Additional research is needed on the efficacy of the mHealth interventions and their side effects, including inequalities because of the digital divide, addictive internet behavior, lack of trust in mHealth, anonymity issues, risks and biases for user groups and social contexts, and ethical implications. Further research is also required to integrate and test psychological theories to improve the impact of mHealth and adherence to the e-mental health interventions.

Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review.

BACKGROUND: Chatbots have been used in the last decade to improve access to mental health care services. Perceptions and opinions of patients influence the adoption of chatbots for health care. Many studies have been conducted to assess the perceptions and opinions of patients about mental health chatbots. To the best of our knowledge, there has been no review of the evidence surrounding perceptions and opinions of patients about mental health chatbots. OBJECTIVE: This study aims to conduct a scoping review of the perceptions and opinions of patients about chatbots for mental health. METHODS: The scoping review was carried out in line with the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension for scoping reviews guidelines. Studies were identified by searching 8 electronic databases (eg, MEDLINE and Embase) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. In total, 2 reviewers independently selected studies and extracted data from the included studies. Data were synthesized using thematic analysis. RESULTS: Of 1072 citations retrieved, 37 unique studies were included in the review. The thematic analysis generated 10 themes from the findings of the studies: usefulness, ease of use, responsiveness, understandability, acceptability, attractiveness, trustworthiness, enjoyability, content, and comparisons. CONCLUSIONS: The results demonstrated overall positive perceptions and opinions of patients about chatbots for mental health. Important issues to be addressed in the future are the linguistic capabilities of the chatbots: they have to be able to deal adequately with unexpected user input, provide high-quality responses, and have to show high variability in responses. To be useful for clinical practice, we have to find ways to harmonize chatbot content with individual treatment recommendations, that is, a personalization of chatbot conversations is required.

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review.

BACKGROUND: Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. OBJECTIVE: This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. METHODS: Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. RESULTS: Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). CONCLUSIONS: The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies.

Social media and internet-based data in global systems for public health surveillance: a systematic review.

CONTEXT: The exchange of health information on the Internet has been heralded as an opportunity to improve public health surveillance. In a field that has traditionally relied on an established system of mandatory and voluntary reporting of known infectious diseases by doctors and laboratories to governmental agencies, innovations in social media and so-called user-generated information could lead to faster recognition of cases of infectious disease. More direct access to such data could enable surveillance epidemiologists to detect potential public health threats such as rare, new diseases or early-level warnings for epidemics. But how useful are data from social media and the Internet, and what is the potential to enhance surveillance? The challenges of using these emerging surveillance systems for infectious disease epidemiology, including the specific resources needed, technical requirements, and acceptability to public health practitioners and policymakers, have wide-reaching implications for public health surveillance in the 21st century. METHODS: This article divides public health surveillance into indicator-based surveillance and event-based surveillance and provides an overview of each. We did an exhaustive review of published articles indexed in the databases PubMed, Scopus, and Scirus between 1990 and 2011 covering contemporary event-based systems for infectious disease surveillance. FINDINGS: Our literature review uncovered no event-based surveillance systems currently used in national surveillance programs. While much has been done to develop event-based surveillance, the existing systems have limitations. Accordingly, there is a need for further development of automated technologies that monitor health-related information on the Internet, especially to handle large amounts of data and to prevent information overload. The dissemination to health authorities of new information about health events is not always efficient and could be improved. No comprehensive evaluations show whether event-based surveillance systems have been integrated into actual epidemiological work during real-time health events. CONCLUSIONS: The acceptability of data from the Internet and social media as a regular part of public health surveillance programs varies and is related to a circular challenge: the willingness to integrate is rooted in a lack of effectiveness studies, yet such effectiveness can be proved only through a structured evaluation of integrated systems. Issues related to changing technical and social paradigms in both individual perceptions of and interactions with personal health data, as well as social media and other data from the Internet, must be further addressed before such information can be integrated into official surveillance systems.

Towards a Reporting Guideline for Studies on Information Extraction from Clinical Texts.

BACKGROUND: The rapid technical progress in the domain of clinical Natural Language Processing and information extraction (IE) has resulted in challenges concerning the comparability and replicability of studies. AIM: This paper proposes a reporting guideline to standardize the description of methodologies and outcomes for studies involving IE from clinical texts. METHODS: The guideline is developed based on the experiences gained from data extraction for a previously conducted scoping review on IE from free-text radiology reports including 34 studies. RESULTS: The guideline comprises the five top-level categories information model, architecture, data, annotation, and outcomes. In total, we define 28 aspects to be reported on in IE studies related to these categories. CONCLUSIONS: The proposed guideline is expected to set a standard for reporting in studies describing IE from clinical text and promote uniformity across the research field. Expected future technological advancements may make regular updates of the guideline necessary. In future research, we plan to develop a taxonomy that clearly defines corresponding value sets as well as integrating both this guideline and the taxonomy by following a consensus-based methodology.

Analysis of Critical Incident Reports Using Natural Language Processing.

A Critical Incident Reporting System (CIRS) collects anecdotal reports from employees, which serve as a vital source of information about incidents that could potentially harm patients. OBJECTIVES: To demonstrate how natural language processing (NLP) methods can help in retrieving valuable information from such incident data. METHODS: We analyzed frequently occurring terms and sentiments as well as topics in data from the Swiss National CIRRNET database from 2006 to 2023 using NLP and BERTopic modelling. RESULTS: We grouped the topics into 10 major themes out of which 6 are related to medication. Overall, they reflect the global trends in adverse events in healthcare (surgical errors, venous thromboembolism, falls). Additionally, we identified errors related to blood testing, COVID-19, handling patients with diabetes and pediatrics. 40-50% of the messages are written in a neutral tone, 30-40% in a negative tone. CONCLUSION: The analysis of CIRS messages using text analysis tools helped in getting insights into common sources of critical incidents in Swiss healthcare institutions. In future work, we want to study more closely the relations, for example between sentiment and topics.

Identifying Reusable Core Assets of Digital Health Apps.

Despite their variability, Digital Health Apps (DHAs) typically share functionalities (i.e. core assets) and can thus be considered as a family of similar products with unique features adapted to specific use cases. OBJECTIVE: We aim to identify and model reusable core assets to facilitate the development of a number of similar, but adapted DHAs in the context of an initial product line engineering approach. METHODS: To identify core assets, we apply a systematic analysis of six exemplary state-of-the-art DHAs. In an iterative process, they were modeled in a feature model. RESULTS: We identified 14 core assets of DHAs out of which six are mandatory (i.e. required by each DHA) and eight are optional core assets (i.e. required by most DHAs, depending on the app complexity). CONCLUSIONS: We found that DHAs share common functionalities that could contribute to a more efficient development of the DHA, especially in terms of time and cost savings.

Large Language Model-Based Evaluation of Medical Question Answering Systems: Algorithm Development and Case Study.

BACKGROUND: Healthcare systems are increasingly resource constrained, leaving less time for important patient-provider interactions. Conversational agents (CAs) could be used to support the provision of information and to answer patients' questions. However, information must be accessible to a variety of patient populations, which requires understanding questions expressed at different language levels. METHODS: This study describes the use of Large Language Models (LLMs) to evaluate predefined medical content in CAs across patient populations. These simulated populations are characterized by a range of health literacy. The evaluation framework includes both fully automated and semi-automated procedures to assess the performance of a CA. RESULTS: A case study in the domain of mammography shows that LLMs can simulate questions from different patient populations. However, the accuracy of the answers provided varies depending on the level of health literacy. CONCLUSIONS: Our scalable evaluation framework enables the simulation of patient populations with different health literacy levels and helps to evaluate domain specific CAs, thus promoting their integration into clinical practice. Future research aims to extend the framework to CAs without predefined content and to apply LLMs to adapt medical information to the specific (health) literacy level of the user.

A scoping review of large language model based approaches for information extraction from radiology reports.

Radiological imaging is a globally prevalent diagnostic method, yet the free text contained in radiology reports is not frequently used for secondary purposes. Natural Language Processing can provide structured data retrieved from these reports. This paper provides a summary of the current state of research on Large Language Model (LLM) based approaches for information extraction (IE) from radiology reports. We conduct a scoping review that follows the PRISMA-ScR guideline. Queries of five databases were conducted on August 1st 2023. Among the 34 studies that met inclusion criteria, only pre-transformer and encoder-based models are described. External validation shows a general performance decrease, although LLMs might improve generalizability of IE approaches. Reports related to CT and MRI examinations, as well as thoracic reports, prevail. Most common challenges reported are missing validation on external data and augmentation of the described methods. Different reporting granularities affect the comparability and transparency of approaches.

Success Factors for Usability Testing of Digital Health Solutions.

BACKGROUND: The design and development of patient-centered digital health solutions requires user involvement, for example through usability testing. Although there are guidelines for conducting usability tests, there is a lack of knowledge about the technical, human, and organizational factors that influence the success of the tests. OBJECTIVE: To summarize the success factors of usability testing in the context of patient-centered digital health solutions. METHOD: We considered three case studies and collected experiences related to time management, relevance of results and challenges encountered. RESULTS: Success factors relate to participant privacy and data protection, test environment setup, device and application readiness, user comfort and accessibility, test tools and procedures, and adaptability to user limitations. CONCLUSIONS: Small organizational and technical details can have a big impact on the outcome of a usability test. Considering the aspects mentioned in this paper will not only save resources but also the trust of the participating patients.

Understanding Emotional Dynamics in Autism Social Media Communities.

Searches for autism on social media have soared, making it a top topic. Social media posts convey not only plain text, but also sentiments and emotions that provide insight into the experiences of the autism community. While sentiment analysis categorizes overall sentiment, emotion analysis provides nuanced insights into specific emotional states. The objective of this study is to identify emotions in posts related to autism and compare the emotions specifically contained in posts that include the hashtag #ActuallyAutistic with those that do not. METHODS: We extracted a sample of X' posts related to autism and used DistilBERT to assign one out of six emotions (sadness, joy, love, anger, fear, surprise) to each post. RESULTS: We have analyzed a total of 414,287 posts, 98,602 (23.8%) of those included the hashtag #ActuallyAutistic. The most common expressed emotion was joy, which was expressed in 52.5% of the posts, followed by sadness, identified in 28.6% of the posts. 12% of the posts expressed fear, 4.9% reflected anger, 1.1% showed love, and 0.9% expressed surprise. Posts tagged as #ActuallyAutistic showed less joy (27.1% vs. 60.4% in posts without this hashtag, p<0.001) and more sadness (52.7% vs. 21.1% in those without the hashtag, p<0.001). CONCLUSIONS: The use of the hashtag #ActuallyAutistic is associated with a different emotional tone, characterized by less joy and more sadness. These results suggest the need for greater support and acceptance towards the autistic community, both online and in society in general. Insights from our study can be valuable for policy makers, health, educational or other programmes aiming at enhancing well-being, inclusiveness, improve services, and create a more compassionate and understanding atmosphere for autistic people.

Human Factors in AI-Driven Digital Solutions for Increasing Physical Activity: Scoping Review.

BACKGROUND: Artificial intelligence (AI) has the potential to enhance physical activity (PA) interventions. However, human factors (HFs) play a pivotal role in the successful integration of AI into mobile health (mHealth) solutions for promoting PA. Understanding and optimizing the interaction between individuals and AI-driven mHealth apps is essential for achieving the desired outcomes. OBJECTIVE: This study aims to review and describe the current evidence on the HFs in AI-driven digital solutions for increasing PA. METHODS: We conducted a scoping review by searching for publications containing terms related to PA, HFs, and AI in the titles and abstracts across 3 databases-PubMed, Embase, and IEEE Xplore-and Google Scholar. Studies were included if they were primary studies describing an AI-based solution aimed at increasing PA, and results from testing the solution were reported. Studies that did not meet these criteria were excluded. Additionally, we searched the references in the included articles for relevant research. The following data were extracted from included studies and incorporated into a qualitative synthesis: bibliographic information, study characteristics, population, intervention, comparison, outcomes, and AI-related information. The certainty of the evidence in the included studies was evaluated using GRADE (Grading of Recommendations Assessment, Development, and Evaluation). RESULTS: A total of 15 studies published between 2015 and 2023 involving 899 participants aged approximately between 19 and 84 years, 60.7% (546/899) of whom were female participants, were included in this review. The interventions lasted between 2 and 26 weeks in the included studies. Recommender systems were the most commonly used AI technology in digital solutions for PA (10/15 studies), followed by conversational agents (4/15 studies). User acceptability and satisfaction were the HFs most frequently evaluated (5/15 studies each), followed by usability (4/15 studies). Regarding automated data collection for personalization and recommendation, most systems involved fitness trackers (5/15 studies). The certainty of the evidence analysis indicates moderate certainty of the effectiveness of AI-driven digital technologies in increasing PA (eg, number of steps, distance walked, or time spent on PA). Furthermore, AI-driven technology, particularly recommender systems, seems to positively influence changes in PA behavior, although with very low certainty evidence. CONCLUSIONS: Current research highlights the potential of AI-driven technologies to enhance PA, though the evidence remains limited. Longer-term studies are necessary to assess the sustained impact of AI-driven technologies on behavior change and habit formation. While AI-driven digital solutions for PA hold significant promise, further exploration into optimizing AI's impact on PA and effectively integrating AI and HFs is crucial for broader benefits. Thus, the implications for innovation management involve conducting long-term studies, prioritizing diversity, ensuring research quality, focusing on user experience, and understanding the evolving role of AI in PA promotion.