Redefining Virtual Assistants in Health Care: The Future With Large Language Models.

This editorial explores the evolving and transformative role of large language models (LLMs) in enhancing the capabilities of virtual assistants (VAs) in the health care domain, highlighting recent research on the performance of VAs and LLMs in health care information sharing. Focusing on recent research, this editorial unveils the marked improvement in the accuracy and clinical relevance of responses from LLMs, such as GPT-4, compared to current VAs, especially in addressing complex health care inquiries, like those related to postpartum depression. The improved accuracy and clinical relevance with LLMs mark a paradigm shift in digital health tools and VAs. Furthermore, such LLM applications have the potential to dynamically adapt and be integrated into existing VA platforms, offering cost-effective, scalable, and inclusive solutions. These suggest a significant increase in the applicable range of VA applications, as well as the increased value, risk, and impact in health care, moving toward more personalized digital health ecosystems. However, alongside these advancements, it is necessary to develop and adhere to ethical guidelines, regulatory frameworks, governance principles, and privacy and safety measures. We need a robust interdisciplinary collaboration to navigate the complexities of safely and effectively integrating LLMs into health care applications, ensuring that these emerging technologies align with the diverse needs and ethical considerations of the health care domain.

Design and Evaluation Challenges of Conversational Agents in Health Care and Well-being: Selective Review Study.

BACKGROUND: Health care and well-being are 2 main interconnected application areas of conversational agents (CAs). There is a significant increase in research, development, and commercial implementations in this area. In parallel to the increasing interest, new challenges in designing and evaluating CAs have emerged. OBJECTIVE: This study aims to identify key design, development, and evaluation challenges of CAs in health care and well-being research. The focus is on the very recent projects with their emerging challenges. METHODS: A review study was conducted with 17 invited studies, most of which were presented at the ACM (Association for Computing Machinery) CHI 2020 conference workshop on CAs for health and well-being. Eligibility criteria required the studies to involve a CA applied to a health or well-being project (ongoing or recently finished). The participating studies were asked to report on their projects' design and evaluation challenges. We used thematic analysis to review the studies. RESULTS: The findings include a range of topics from primary care to caring for older adults to health coaching. We identified 4 major themes: (1) Domain Information and Integration, (2) User-System Interaction and Partnership, (3) Evaluation, and (4) Conversational Competence. CONCLUSIONS: CAs proved their worth during the pandemic as health screening tools, and are expected to stay to further support various health care domains, especially personal health care. Growth in investment in CAs also shows the value as a personal assistant. Our study shows that while some challenges are shared with other CA application areas, safety and privacy remain the major challenges in the health care and well-being domains. An increased level of collaboration across different institutions and entities may be a promising direction to address some of the major challenges that otherwise would be too complex to be addressed by the projects with their limited scope and budget.

Identifying the Factors Influencing Patients' Telehealth Visit Satisfaction: Survey Validation Through a Structural Equation Modeling Approach.

Introduction: The COVID-19 pandemic accelerated the adoption of telehealth as an alternative to in-person hospital visits. To understand the factors impacting the quality of telehealth services, there is a need for validated survey instruments and conceptual frameworks. The objective of this study is to validate a telehealth patient satisfaction survey by structural equation modeling (SEM) and determine the relationship between the factors in the proposed telehealth patient satisfaction model (TPSM). Methods: We conducted a cross-sectional survey of pediatric patients and families receiving care from a comprehensive pediatric hospital in the Midwest between September 2020 and January 2021. In total, 2,039 usable responses were collected. We used an SEM approach by performing confirmatory factor analysis with Diagonally Weighted Least Squares modeling and Partial Least Squares-Path Modeling to establish the structural validity and examined the relationships among the constructs of "Admission Process" (AP), "Perceived Quality of Service" (PQS), and "Telehealth Satisfaction" (TS). Results: Participants were predominantly White (75%) and English-speaking (95%) parents (85%) of patients (mean age of patients was 10.2 years old). The survey responses were collected from patients visiting 43 department specialties, whereas 50% were behavioral and occupational therapy patients. The structural model showed that the admission process (AP) had a strong positive impact on perceived quality of service (PQS) (p = 0.67, t = 36.1, p < 0.001). The PQS had a strong positive impact on telehealth satisfaction (TS) (p = 0.66, t = 31.8, p < 0.001). The AP had a low positive direct impact on TS (p = 0.16, t = 7.46, p < 0.05). Overall, AP and PQS explained 61% variances (R2) of TS. Conclusions: We validated a newly proposed TS assessment model by using SEM. The TPSM will inform researchers to better understand the influencing factors in TS and help health care systems to improve telehealth patient satisfaction through a validated model.

Rapid Development of a Telehealth Patient Satisfaction Survey Using a Multi-Stakeholder Approach.

Introduction: The COVID-19 pandemic has hastened the adoption of telehealth and the drastic shift to an unfamiliar process may impose significant impact to the quality-of-care delivery. Many providers are interested in understanding the quality of their telehealth services from the patients' experience. Materials and Methods: A telehealth patient satisfaction survey (TPSS) was developed by using an iterative stakeholder-centered design approach, incorporating elements from validated telemedicine and customer service survey instruments, and meeting the operational needs and constraints. A cross-sectional study design was employed to collect survey responses from patients and families of a large pediatric hospital. Finally, we performed exploratory factor analysis (EFA) to extract latent constructs and factor loadings of the survey items to further explain relationships. Results: A 22-item TPSS closely matched the existing in-person patient satisfaction survey and mapped to a revised SERVPERF conceptual model that was proposed by the interdisciplinary committee. Survey was implemented in the HIPAA-compliant online platform REDCap® with survey link embedded in an automated Epic MyChart (Verona, WI) visit follow-up message. In total, 2,394 survey responses were collected between July 7, 2020, and September 2, 2020. EFA revealed three constructs (with factor loadings >0.30): admission process, perceived quality of services, and telehealth satisfaction. Conclusions: We reported the development of TPSS that met the operational needs of compatibility with existing data and possible comparison to in-person survey. The survey is short and yet covers both the clinical experience and telehealth usability, with acceptable survey validity.

Documented Reasons of Cancellation and Rescheduling of Telehealth Appointments During the Pandemic.

Background and Objective: The COVID-19 pandemic increased the use of telehealth around the world. The aim is to minimize health care service disruption as well as reducing COVID-19 exposure. However, one of the major operational concerns is cancellations and rescheduling (C/Rs). C/Rs may create additional burden and cost to the patient, provider, and the health system. Our aim is to understand the reasons for C/Rs of the telehealth session after the scheduled start time. Materials and Methods: We reviewed electronic health records (EHRs) to identify the C/R reasons for behavioral health and speech language pathology departments. Documented C/Rs in the medical charts were identified from EHR by using a keyword-based and Natural Language Processing (NLP)-supported EHR search engine. From the search results, we randomly selected 200 notes and conducted a thematic analysis. Results: We identified four themes explaining C/R reasons. Most frequent theme was "technicality" (47, 36%), followed by "engagement" (34, 25%), "scheduling" (31, 24%), and "unspecified" (20, 15%). The findings showed that technical reasons are the leading cause of C/Rs, constituting 36% of the cases (95% confidence interval [CI]: 29-43%). Notably, "engagement" constituted a sizeable 25% (95% CI: 19-31%) of C/Rs, as a result of the inability to engage a patient to complete the telehealth session. Conclusions: The study shows that engagement is one of the new challenges to the pediatric telehealth visits. Future studies of new engagement models are needed for the success of telehealth. Our findings will help fill the literature gaps and may help with enhancing the digital experience for both caregivers and providers, reducing wasted time and resources due to preventable C/Rs, improving clinical operation efficiency, and treatment adherence.

Capturing At-Home Health and Care Information for Children With Medical Complexity Using Voice Interactive Technologies: Multi-Stakeholder Viewpoint.

Digital health tools and technologies are transforming health care and making significant impacts on how health and care information are collected, used, and shared to achieve best outcomes. As most of the efforts are still focused on clinical settings, the wealth of health information generated outside of clinical settings is not being fully tapped. This is especially true for children with medical complexity (CMC) and their families, as they frequently spend significant hours providing hands-on medical care within the home setting and coordinating activities among multiple providers and other caregivers. In this paper, a multidisciplinary team of stakeholders discusses the value of health information generated at home, how technology can enhance care coordination, and challenges of technology adoption from a patient-centered perspective. Voice interactive technology has been identified to have the potential to transform care coordination for CMC. This paper shares opinions on the promises, limitations, recommended approaches, and challenges of adopting voice technology in health care, especially for the targeted patient population of CMC.

Proposing an Ecosystem of Digital Health Solutions for Teens With Chronic Conditions Transitioning to Self-Management and Independence: Exploratory Qualitative Study.

BACKGROUND: Chronic disease management is critical to quality of life for both teen patients with chronic conditions and their caregivers. However, current literature is largely limited to a specific digital health tool, method, or approach to manage a specific disease. Guiding principles on how to use digital tools to support the transition to independence are rare. Considering the physiological, psychological, and environmental changes that teens experience, the issues surrounding the transition to independence are worth investigating to develop a deeper understanding to inform future strategies for digital interventions. OBJECTIVE: The purpose of this study was to inform the design of digital health solutions by systematically identifying common challenges among teens and caregivers living with chronic diseases. METHODS: Chronically ill teens (n=13) and their caregivers (n=13) were interviewed individually and together as a team. Verbal and projective techniques were used to examine teens' and caregivers' concerns in-depth. The recorded and transcribed responses were thematically analyzed to identify and organize the identified patterns. RESULTS: Teens and their caregivers identified 10 challenges and suggested technological solutions. Recognized needs for social support, access to medical education, symptom monitoring, access to health care providers, and medical supply management were the predominant issues. The envisioned ideal transition included a 5-component solution ecosystem in the transition to independence for teens. CONCLUSIONS: This novel study systematically summarizes the challenges, barriers, and technological solutions for teens with chronic conditions and their caregivers as teens transition to independence. A new solution ecosystem based on the 10 identified challenges would guide the design of future implementations to test and validate the effectiveness of the proposed 5-component ecosystem.

Evaluation of a Digital Scribe: Conversation Summarization for Emergency Department Consultation Calls.

OBJECTIVE: We present a proof-of-concept digital scribe system as an Emergency Department (ED) consultation call-based clinical conversation summarization pipeline to support clinical documentation, and report its performance. MATERIALS AND METHODS: We use four pre-trained large language models to establish the digital scribe system: T5-small, T5-base, PEGASUS-PubMed, and BART-Large-CNN via zero-shot and fine-tuning approaches. Our dataset includes 100 referral conversations among ED clinicians and medical records. We report the ROUGE-1, ROUGE-2, and ROUGE-L to compare model performance. In addition, we annotated transcriptions to assess the quality of generated summaries. RESULTS: The fine-tuned BART-Large-CNN model demonstrates greater performance in summarization tasks with the highest ROUGE scores (F1ROUGE-1=0.49, F1ROUGE-2=0.23, F1ROUGE-L=0.35) scores. In contrast, PEGASUS-PubMed lags notably (F1ROUGE-1=0.28, F1ROUGE-2=0.11, F1ROUGE-L=0.22). BART-Large-CNN's performance decreases by more than 50% with the zero-shot approach. Annotations show that BART-Large-CNN performs 71.4% recall in identifying key information and a 67.7% accuracy rate. DISCUSSION: The BART-Large-CNN model demonstrates a high level of understanding of clinical dialogue structure, indicated by its performance with and without fine-tuning. Despite some instances of high recall, there is variability in the model's performance, particularly in achieving consistent correctness, suggesting room for refinement. The model's recall ability varies across different information categories. CONCLUSION: The study provides evidence towards the potential of AI-assisted tools in assisting clinical documentation. Future work is suggested on expanding the research scope with additional language models and hybrid approaches, and comparative analysis to measure documentation burden and human factors.

Acceptance of voice assistant technology in dental practice: A cross sectional study with dentists and validation using structural equation modeling.

Voice assistant technologies (VAT) has been part of our daily lives, as a virtual assistant to complete requested tasks. The integration of VAT in dental offices has the potential to augment productivity and hygiene practices. Prior to the adoption of such innovations in dental settings, it is crucial to evaluate their applicability. This study aims to assess dentists' perceptions and the factors influencing their intention to use VAT in a clinical setting. A survey and research model were designed based on an extended Unified Theory of Acceptance and Use of Technology (UTAUT). The survey was sent to 7,544 Ohio-licensed dentists through email. The data was analyzed and reported using descriptive statistics, model reliability testing, and partial least squares regression (PLSR) to explain dentists' behavioral intention (BI) to use VAT. In total, 257 participants completed the survey. The model accounted for 74.2% of the variance in BI to use VAT. Performance expectancy and perceived enjoyment had significant positive influence on BI to use VAT. Perceived risk had significant negative influence on BI to use VAT. Self-efficacy had significantly influenced perceived enjoyment, accounting for 35.5% of the variance of perceived enjoyment. This investigation reveals that performance efficiency and user enjoyment are key determinants in dentists' decision to adopt VAT. Concerns regarding the privacy of VAT also play a crucial role in its acceptance. This study represents the first documented inquiry into dentists' reception of VAT, laying groundwork for future research and implementation strategies.

Mobile Application Measurement of Menstrual Cycle Characteristics and Their Association with Dysmenorrhea and Activity Limitation in Early Adolescents.

OBJECTIVE: Real-time tracking of menstrual bleeding is a barrier to research due to limitations with traditional data collection tools. This prospective cohort study utilized a mobile application (TDot app) in young adolescents aged 10-14 years to assess the relationship between heavy menstrual bleeding (HMB), dysmenorrhea, and activity limitation. METHODS: Menstrual cycles were captured over six months in real-time using the Pictorial Blood loss Assessment Chart (PBAC). A median PBAC score of >100 was used to identify participants with HMB. Participants also completed a modified WaLIDD (Working ability, Location, Intensity, Days of pain, Dysmenorrhea) scale. Impact of menses on daily activities was collected for each cycle. RESULTS: A total of 160 participants enrolled and 100 (63%) participants with ≥3 cycles recorded in the mobile app were analyzed. HMB was noted in 41% of participants. Median modified WaLIDD score was significantly higher in participants with HMB than those without HMB (p=0.01). No significant differences were found in activity limitations between participants with and without HMB (p=0.34). Median modified WaLIDD score for participants with activity limitation was significantly higher than those without activity limitation (p=0.01). CONCLUSION: Utilizing mobile app technology, we were able to gather real-time menstrual outcome data from young adolescents on heaviness of flow, dysmenorrhea and activity limitations. While we did not find that patients with HMB were more likely to have activity limitations, we did find that those with limitations had modestly higher dysmenorrhea scores. Future studies should focus on identifying additional variables that impact activity limitation during menstruation.

Artificial intelligence in healthcare: Complementing, not replacing, doctors and healthcare providers.

The utilization of artificial intelligence (AI) in clinical practice has increased and is evidently contributing to improved diagnostic accuracy, optimized treatment planning, and improved patient outcomes. The rapid evolution of AI, especially generative AI and large language models (LLMs), have reignited the discussions about their potential impact on the healthcare industry, particularly regarding the role of healthcare providers. Concerning questions, "can AI replace doctors?" and "will doctors who are using AI replace those who are not using it?" have been echoed. To shed light on this debate, this article focuses on emphasizing the augmentative role of AI in healthcare, underlining that AI is aimed to complement, rather than replace, doctors and healthcare providers. The fundamental solution emerges with the human-AI collaboration, which combines the cognitive strengths of healthcare providers with the analytical capabilities of AI. A human-in-the-loop (HITL) approach ensures that the AI systems are guided, communicated, and supervised by human expertise, thereby maintaining safety and quality in healthcare services. Finally, the adoption can be forged further by the organizational process informed by the HITL approach to improve multidisciplinary teams in the loop. AI can create a paradigm shift in healthcare by complementing and enhancing the skills of healthcare providers, ultimately leading to improved service quality, patient outcomes, and a more efficient healthcare system.

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden.

Objective: We present a proof-of-concept digital scribe system as an ED clinical conversation summarization pipeline and report its performance. Materials and Methods: We use four pre-trained large language models to establish the digital scribe system: T5-small, T5-base, PEGASUS-PubMed, and BART-Large-CNN via zero-shot and fine-tuning approaches. Our dataset includes 100 referral conversations among ED clinicians and medical records. We report the ROUGE-1, ROUGE-2, and ROUGE-L to compare model performance. In addition, we annotated transcriptions to assess the quality of generated summaries. Results: The fine-tuned BART-Large-CNN model demonstrates greater performance in summarization tasks with the highest ROUGE scores (F1ROUGE-1=0.49, F1ROUGE-2=0.23, F1ROUGE-L=0.35) scores. In contrast, PEGASUS-PubMed lags notably (F1ROUGE-1=0.28, F1ROUGE-2=0.11, F1ROUGE-L=0.22). BART-Large-CNN's performance decreases by more than 50% with the zero-shot approach. Annotations show that BART-Large-CNN performs 71.4% recall in identifying key information and a 67.7% accuracy rate. Discussion: The BART-Large-CNN model demonstrates a high level of understanding of clinical dialogue structure, indicated by its performance with and without fine-tuning. Despite some instances of high recall, there is variability in the model's performance, particularly in achieving consistent correctness, suggesting room for refinement. The model's recall ability varies across different information categories. Conclusion: The study provides evidence towards the potential of AI-assisted tools in reducing clinical documentation burden. Future work is suggested on expanding the research scope with larger language models, and comparative analysis to measure documentation efforts and time.

Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data.

BACKGROUND: Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient's journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. OBJECTIVE: Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. METHODS: We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. RESULTS: In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. CONCLUSIONS: The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data).

Identification of Child Survivors of Sex Trafficking From Electronic Health Records: An Artificial Intelligence Guided Approach.

Survivors of child sex trafficking (SCST) experience high rates of adverse health outcomes. Amidst the duration of their victimization, survivors regularly seek healthcare yet fail to be identified. This study sought to utilize artificial intelligence (AI) to identify SCST and describe the elements of their healthcare presentation. An AI-supported keyword search was conducted to identify SCST within the electronic medical records (EMR) of ∼1.5 million patients at a large midwestern pediatric hospital. Descriptive analyses were used to evaluate associated diagnoses and clinical presentation. A sex trafficking-related keyword was identified in .18% of patient charts. Among this cohort, the most common associated diagnostic codes were for Confirmed Sexual/Physical Assault; Trauma and Stress-Related Disorders; Depressive Disorders; Anxiety Disorders; and Suicidal Ideation. Our findings are consistent with the myriad of known adverse physical and psychological outcomes among SCST and illuminate the future potential of AI technology to improve screening and research efforts surrounding all aspects of this vulnerable population.

Detecting Clinically Relevant Emotional Distress and Functional Impairment in Children and Adolescents: Protocol for an Automated Speech Analysis Algorithm Development Study.

BACKGROUND: Even before the onset of the COVID-19 pandemic, children and adolescents were experiencing a mental health crisis, partly due to a lack of quality mental health services. The rate of suicide for Black youth has increased by 80%. By 2025, the health care system will be short of 225,000 therapists, further exacerbating the current crisis. Therefore, it is of utmost importance for providers, schools, youth mental health, and pediatric medical providers to integrate innovation in digital mental health to identify problems proactively and rapidly for effective collaboration with other health care providers. Such approaches can help identify robust, reproducible, and generalizable predictors and digital biomarkers of treatment response in psychiatry. Among the multitude of digital innovations to identify a biomarker for psychiatric diseases currently, as part of the macrolevel digital health transformation, speech stands out as an attractive candidate with features such as affordability, noninvasive, and nonintrusive. OBJECTIVE: The protocol aims to develop speech-emotion recognition algorithms leveraging artificial intelligence/machine learning, which can establish a link between trauma, stress, and voice types, including disrupting speech-based characteristics, and detect clinically relevant emotional distress and functional impairments in children and adolescents. METHODS: Informed by theoretical foundations (the Theory of Psychological Trauma Biomarkers and Archetypal Voice Categories), we developed our methodology to focus on 5 emotions: anger, happiness, fear, neutral, and sadness. Participants will be recruited from 2 local mental health centers that serve urban youths. Speech samples, along with responses to the Symptom and Functioning Severity Scale, Patient Health Questionnaire 9, and Adverse Childhood Experiences scales, will be collected using an Android mobile app. Our model development pipeline is informed by Gaussian mixture model (GMM), recurrent neural network, and long short-term memory. RESULTS: We tested our model with a public data set. The GMM with 128 clusters showed an evenly distributed accuracy across all 5 emotions. Using utterance-level features, GMM achieved an accuracy of 79.15% overall, while frame selection increased accuracy to 85.35%. This demonstrates that GMM is a robust model for emotion classification of all 5 emotions and that emotion frame selection enhances accuracy, which is significant for scientific evaluation. Recruitment and data collection for the study were initiated in August 2021 and are currently underway. The study results are likely to be available and published in 2024. CONCLUSIONS: This study contributes to the literature as it addresses the need for speech-focused digital health tools to detect clinically relevant emotional distress and functional impairments in children and adolescents. The preliminary results show that our algorithm has the potential to improve outcomes. The findings will contribute to the broader digital health transformation. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/46970.

Improving medication adherence monitoring and clinical outcomes through mHealth: A randomized controlled trial protocol in pediatric stem cell transplant.

Medication non-adherence rates in children range between 50% and 80% in the United States. Due to multifaceted outpatient routines, children receiving hematopoietic stem cell transplant (HCT) are at especially high risk of non-adherence, which can be life-threatening. Although digital health interventions have been effective in improving non-adherence in many pediatric conditions, limited research has examined their benefits among families of children receiving HCT. To address this gap, we created the BMT4me© mobile health app, an innovative intervention serving as a "virtual assistant" to send medication-taking reminders for caregivers and to track, in real-time, the child's medication taking, barriers to missed doses, symptoms or side effects, and other notes regarding their child's treatment. In this randomized controlled trial, caregivers will be randomized to either the control (standard of care) group or the intervention (BMT4me© app) group at initial discharge post-HCT. Both groups will receive an electronic adherence monitoring device (i.e., medication event monitoring system "MEMS" cap, Medy Remote Patient Management "MedyRPM" medication adherence box) to store their child's immunosuppressant medication. Caregivers who agree to participate will be asked to complete enrollment, weekly, and monthly parent-proxy measures of their child's medication adherence until the child reaches Day 100 or complete taper from immunosuppression. Caregivers will also participate in a 15 to 30-minute exit interview at the conclusion of the study. Descriptive statistics and correlations will be used to assess phone activity and use behavior over time. Independent samples t-tests will examine the efficacy of the intervention to improve adherence monitoring and reduce readmission rates. The primary expected outcome of this study is that the BMT4me© app will improve the real-time monitoring and medication adherence in children receiving hematopoietic stem cell transplant following discharge, thus improving clinical outcomes.

Objectively Quantifying Pediatric Psychiatric Severity Using Artificial Intelligence, Voice Recognition Technology, and Universal Emotions: Pilot Study for Artificial Intelligence-Enabled Innovation to Address Youth Mental Health Crisis.

BACKGROUND: Providing Psychotherapy, particularly for youth, is a pressing challenge in the health care system. Traditional methods are resource-intensive, and there is a need for objective benchmarks to guide therapeutic interventions. Automated emotion detection from speech, using artificial intelligence, presents an emerging approach to address these challenges. Speech can carry vital information about emotional states, which can be used to improve mental health care services, especially when the person is suffering. OBJECTIVE: This study aims to develop and evaluate automated methods for detecting the intensity of emotions (anger, fear, sadness, and happiness) in audio recordings of patients' speech. We also demonstrate the viability of deploying the models. Our model was validated in a previous publication by Alemu et al with limited voice samples. This follow-up study used significantly more voice samples to validate the previous model. METHODS: We used audio recordings of patients, specifically children with high adverse childhood experience (ACE) scores; the average ACE score was 5 or higher, at the highest risk for chronic disease and social or emotional problems; only 1 in 6 have a score of 4 or above. The patients' structured voice sample was collected by reading a fixed script. In total, 4 highly trained therapists classified audio segments based on a scoring process of 4 emotions and their intensity levels for each of the 4 different emotions. We experimented with various preprocessing methods, including denoising, voice-activity detection, and diarization. Additionally, we explored various model architectures, including convolutional neural networks (CNNs) and transformers. We trained emotion-specific transformer-based models and a generalized CNN-based model to predict emotion intensities. RESULTS: The emotion-specific transformer-based model achieved a test-set precision and recall of 86% and 79%, respectively, for binary emotional intensity classification (high or low). In contrast, the CNN-based model, generalized to predict the intensity of 4 different emotions, achieved test-set precision and recall of 83% for each. CONCLUSIONS: Automated emotion detection from patients' speech using artificial intelligence models is found to be feasible, leading to a high level of accuracy. The transformer-based model exhibited better performance in emotion-specific detection, while the CNN-based model showed promise in generalized emotion detection. These models can serve as valuable decision-support tools for pediatricians and mental health providers to triage youth to appropriate levels of mental health care services. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR1-10.2196/51912.

Operationalizing and Implementing Pretrained, Large Artificial Intelligence Linguistic Models in the US Health Care System: Outlook of Generative Pretrained Transformer 3 (GPT-3) as a Service Model.

Generative pretrained transformer models have been popular recently due to their enhanced capabilities and performance. In contrast to many existing artificial intelligence models, generative pretrained transformer models can perform with very limited training data. Generative pretrained transformer 3 (GPT-3) is one of the latest releases in this pipeline, demonstrating human-like logical and intellectual responses to prompts. Some examples include writing essays, answering complex questions, matching pronouns to their nouns, and conducting sentiment analyses. However, questions remain with regard to its implementation in health care, specifically in terms of operationalization and its use in clinical practice and research. In this viewpoint paper, we briefly introduce GPT-3 and its capabilities and outline considerations for its implementation and operationalization in clinical practice through a use case. The implementation considerations include (1) processing needs and information systems infrastructure, (2) operating costs, (3) model biases, and (4) evaluation metrics. In addition, we outline the following three major operational factors that drive the adoption of GPT-3 in the US health care system: (1) ensuring Health Insurance Portability and Accountability Act compliance, (2) building trust with health care providers, and (3) establishing broader access to the GPT-3 tools. This viewpoint can inform health care practitioners, developers, clinicians, and decision makers toward understanding the use of the powerful artificial intelligence tools integrated into hospital systems and health care.

Sociotechnical Factors Affecting Patients' Adoption of Mobile Health Tools: Systematic Literature Review and Narrative Synthesis.

BACKGROUND: Mobile health (mHealth) tools have emerged as a promising health care technology that may contribute to cost savings, better access to care, and enhanced clinical outcomes; however, it is important to ensure their acceptance and adoption to harness this potential. Patient adoption has been recognized as a key challenge that requires further exploration. OBJECTIVE: The aim of this review was to systematically investigate the literature to understand the factors affecting patients' adoption of mHealth tools by considering sociotechnical factors (from technical, social, and health perspectives). METHODS: A structured search was completed following the participants, intervention, comparators, and outcomes framework. We searched the MEDLINE, PubMed, Cochrane Library, and SAGE databases for studies published between January 2011 and July 2021 in the English language, yielding 5873 results, of which 147 studies met the inclusion criteria. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Cochrane Handbook were followed to ensure a systematic process. Extracted data were analyzed using NVivo (QSR International), with thematic analysis and narrative synthesis of emergent themes. RESULTS: The technical factors affecting patients' adoption of mHealth tools were categorized into six key themes, which in turn were divided into 20 subthemes: usefulness, ease of use, data-related, monetary factors, technical issues, and user experience. Health-related factors were categorized into six key themes: the disease or health condition, the care team's role, health consciousness and literacy, health behavior, relation to other therapies, integration into patient journey, and the patients' insurance status. Social and personal factors were divided into three key clusters: demographic factors, personal characteristics, and social and cultural aspects; these were divided into 19 subthemes, highlighting the importance of considering these factors when addressing potential barriers to mHealth adoption and how to overcome them. CONCLUSIONS: This review builds on the growing body of research that investigates patients' adoption of mHealth services and highlights the complexity of the factors affecting adoption, including personal, social, technical, organizational, and health care aspects. We recommend a more patient-centered approach by ensuring the tools' fit into the overall patient journey and treatment plan, emphasizing inclusive design, and warranting comprehensive patient education and support. Moreover, empowering and mobilizing clinicians and care teams, addressing ethical data management issues, and focusing on health care policies may facilitate adoption.