Adapting the Number of Questions Based on Detected Psychological Distress for Cognitive Behavioral Therapy With an Embodied Conversational Agent: Comparative Study.

BACKGROUND: The high prevalence of mental illness is a critical social problem. The limited availability of mental health services is a major factor that exacerbates this problem. One solution is to deliver cognitive behavioral therapy (CBT) using an embodied conversational agent (ECA). ECAs make it possible to provide health care without location or time constraints. One of the techniques used in CBT is Socratic questioning, which guides users to correct negative thoughts. The effectiveness of this approach depends on a therapist's skill to adapt to the user's mood or distress level. However, current ECAs do not possess this skill. Therefore, it is essential to implement this adaptation ability to the ECAs. OBJECTIVE: This study aims to develop and evaluate a method that automatically adapts the number of Socratic questions based on the level of detected psychological distress during a CBT session with an ECA. We hypothesize that this adaptive approach to selecting the number of questions will lower psychological distress, reduce negative emotional states, and produce more substantial cognitive changes compared with a random number of questions. METHODS: In this study, which envisions health care support in daily life, we recruited participants aged from 18 to 65 years for an experiment that involved 2 different conditions: an ECA that adapts a number of questions based on psychological distress detection or an ECA that only asked a random number of questions. The participants were assigned to 1 of the 2 conditions, experienced a single CBT session with an ECA, and completed questionnaires before and after the session. RESULTS: The participants completed the experiment. There were slight differences in sex, age, and preexperimental psychological distress levels between the 2 conditions. The adapted number of questions condition showed significantly lower psychological distress than the random number of questions condition after the session. We also found a significant difference in the cognitive change when the number of questions was adapted based on the detected distress level, compared with when the number of questions was fewer than what was appropriate for the level of distress detected. CONCLUSIONS: The results show that an ECA adapting the number of Socratic questions based on detected distress levels increases the effectiveness of CBT. Participants who received an adaptive number of questions experienced greater reductions in distress than those who received a random number of questions. In addition, the participants showed a greater amount of cognitive change when the number of questions matched the detected distress level. This suggests that adapting the question quantity based on distress level detection can improve the results of CBT delivered by an ECA. These results illustrate the advantages of ECAs, paving the way for mental health care that is more tailored and effective.

Talk with Ted: an embodied conversational agent for caregivers.

Communication is key to the success of any relationship. When it comes to caregivers, having a conversation with a person living with some form of cognitive impairment, such as dementia, can be a struggle. Most people living with dementia experience some form of communication impairment that reduces their ability to express their needs. In this case study, we present the design of an embodied conversation agent (ECA), Ted, designed to educate caregivers about the importance of good communication principles when engaging with people living with dementia. This training tool was trialed and compared to an online training tool, with 23 caregivers divided into two cohorts (12 in the ECA condition, and 11 in the online training tool condition), over a period of 8 weeks using a mixed evaluation approach. Our findings suggest that (a) caregivers developed an emotional connection with the ECA and retained the learning from their interactions with Ted even after 8 weeks had elapsed, (b) caregivers implemented the learnings in their practice, and (c) the changes in care practice were well received by people living with dementia.

Zero-shot style transfer for gesture animation driven by text and speech using adversarial disentanglement of multimodal style encoding.

Modeling virtual agents with behavior style is one factor for personalizing human-agent interaction. We propose an efficient yet effective machine learning approach to synthesize gestures driven by prosodic features and text in the style of different speakers including those unseen during training. Our model performs zero-shot multimodal style transfer driven by multimodal data from the PATS database containing videos of various speakers. We view style as being pervasive; while speaking, it colors the communicative behaviors expressivity while speech content is carried by multimodal signals and text. This disentanglement scheme of content and style allows us to directly infer the style embedding even of a speaker whose data are not part of the training phase, without requiring any further training or fine-tuning. The first goal of our model is to generate the gestures of a source speaker based on the content of two input modalities-Mel spectrogram and text semantics. The second goal is to condition the source speaker's predicted gestures on the multimodal behavior style embedding of a target speaker. The third goal is to allow zero-shot style transfer of speakers unseen during training without re-training the model. Our system consists of two main components: (1) a speaker style encoder network that learns to generate a fixed-dimensional speaker embedding style from a target speaker multimodal data (mel-spectrogram, pose, and text) and (2) a sequence-to-sequence synthesis network that synthesizes gestures based on the content of the input modalities-text and mel-spectrogram-of a source speaker and conditioned on the speaker style embedding. We evaluate that our model is able to synthesize gestures of a source speaker given the two input modalities and transfer the knowledge of target speaker style variability learned by the speaker style encoder to the gesture generation task in a zero-shot setup, indicating that the model has learned a high-quality speaker representation. We conduct objective and subjective evaluations to validate our approach and compare it with baselines.

Capturing Conversational Gestures for Embodied Conversational Agents Using an Optimized Kaneda-Lucas-Tomasi Tracker and Denavit-Hartenberg-Based Kinematic Model.

In order to recreate viable and human-like conversational responses, the artificial entity, i.e., an embodied conversational agent, must express correlated speech (verbal) and gestures (non-verbal) responses in spoken social interaction. Most of the existing frameworks focus on intent planning and behavior planning. The realization, however, is left to a limited set of static 3D representations of conversational expressions. In addition to functional and semantic synchrony between verbal and non-verbal signals, the final believability of the displayed expression is sculpted by the physical realization of non-verbal expressions. A major challenge of most conversational systems capable of reproducing gestures is the diversity in expressiveness. In this paper, we propose a method for capturing gestures automatically from videos and transforming them into 3D representations stored as part of the conversational agent's repository of motor skills. The main advantage of the proposed method is ensuring the naturalness of the embodied conversational agent's gestures, which results in a higher quality of human-computer interaction. The method is based on a Kanade-Lucas-Tomasi tracker, a Savitzky-Golay filter, a Denavit-Hartenberg-based kinematic model and the EVA framework. Furthermore, we designed an objective method based on cosine similarity instead of a subjective evaluation of synthesized movement. The proposed method resulted in a 96% similarity.

Bibliometric Analysis of Chatbots in Health-Trend Shifts and Advancements in Artificial Intelligence for Personalized Conversational Agents.

Bibliometric analysis provides a summary for research reported in scientific literature. This can highlight pattens and trends in academic research areas, and assist in research directions. Recent growing requirements for efficient communications and increased user learning needs in the health domain, has instigated mass exploitation of chatbots. 2148 documents were analysed to show a shift in research focus around the year 2016. The rate of documents produced in the last few years is more than the collective 20 year period, and future outputs may soar. The emergence of machine and deep learning technology with chatbot usage suggested research opportunity to be exploited in techniques which embed advanced AI abilities. Key authors still spearhead the research direction but a new wave of outputs will further disperse topics into advanced techniques such as personalised disease detections and sophisticated interface that significantly mask any artificiality to their composition.

Automatic topic selection for long-term interaction with embodied conversational agents in health coaching: A micro-randomized trial.

INTRODUCTION: Embodied Conversational Agents (ECAs) can be included in health coaching applications as virtual coaches. The engagement with these virtual coaches could be improved by presenting users with tailored coaching dialogues. In this article, we investigate if the suggestion of an automatically tailored topic by an ECA leads to higher engagement by the user and thus longer sessions of interaction. METHODS: A Micro-Randomized Trial (MRT) was conducted in which two types of interaction with an ECA were compared: (a) the coach suggests a relevant topic to discuss, and (b) the coach asks the user to select a topic from a set of options. Every time the user would interact with the ECA, one of those conditions would be randomly selected. Participants interacted in their daily life with the ECA that was part of a multi-agent health coaching application for 4-8 weeks. RESULTS: In two rounds, 82 participants interacted with the micro-randomized coach a total of 1011 times. Interactions in which the coach took the initiative were found to be of equal length as interactions in which the user was allowed to choose the topic, and the acceptance of topic suggestions was high (71.1% overall, 75.8% for coaching topics). CONCLUSION: Tailoring coaching conversations with ECAs by letting the coach automatically suggest a topic that is tailored to the user is perceived as a natural variation in the flow of interaction. Future research could focus on improving the novel coaching engine component that supports the topic selection process for these suggestions or on investigating how the amount of initiative and coaching approach by the ECA could be tailored.

Aging Adults' Motivation to Use Embodied Conversational Agents in Instrumental Activities of Daily Living: Results of Latent Profile Analysis.

The rapidly increasing share of ageing adults in the population drives the need and interest in assistive technology, as it has the potential to support ageing individuals in living independently and safely. However, technological development rarely reflects how needs, preferences, and interests develop in different ways while ageing. It often follows the strategy of "what is possible" rather than "what is needed" and "what preferred". As part of personalized assistive technology, embodied conversational agents (ECAs) can offer mechanisms to adapt the technological advances with the stakeholders' expectations. The present study explored the motivation among ageing adults regarding technology use in multiple domains of activities of daily living. Participants responded to the questionnaire on the perceived importance of instrumental activities of daily living and acceptance of the idea of using ECAs to support them. Latent profile analysis revealed four profiles regarding the motivation to use ECAs (i.e., a low motivation profile, two selective motivation profiles with an emphasis on physical and psychological well-being, and a high motivation profile). Profiles were compared in terms of their acceptance of ECA usage in various life domains. The results increase the knowledge needed in the development of assistive technology adapted to the expectations of ageing adults.

User Requirements Analysis of an Embodied Conversational Agent for Coaching Older Adults to Choose Active and Healthy Ageing Behaviors during the Transition to Retirement: A Cross-National User Centered Design Study.

BACKGROUND: Retirement is recognized as a factor influencing the ageing process. Today, virtual health coaching systems can play a pivotal role in supporting older adults' active and healthy ageing. This study wants to answer two research questions: (1) What are the user requirements of a virtual coach (VC) based on an Embodied Conversational Agent (ECA) for motivating older adults in transition to retirement to adopt a healthy lifestyle? (2) How could a VC address the active and healthy ageing dimensions, even during COVID-19 times? METHODS: Two-wave focus-groups with 60 end-users aged 55 and over and 27 follow-up telephone interviews were carried out in Austria, Italy and the Netherlands in 2019-2020. Qualitative data were analysed by way of framework analysis. RESULTS: End-users suggest the VC should motivate older workers and retirees to practice physical activity, maintain social contacts and emotional well-being. The ECA should be reactive, customizable, expressive, sympathetic, not directive nor patronizing, with a pleasant and motivating language. The COVID-19 outbreak increased the users' need for functions boosting community relationships and promoting emotional well-being. CONCLUSIONS: the VC can address the active and healthy ageing paradigm by increasing the chances of doing low-cost healthy activities at any time and in any place.

Friends from the Future: A Scoping Review of Research into Robots and Computer Agents to Combat Loneliness in Older People.

BACKGROUND AND AIM: Loneliness is a common problem in older adults and contributes to poor health. This scoping review aimed to synthesize and report evidence on the effectiveness of interventions using social robots or computer agents to reduce loneliness in older adults and to explore intervention strategies. METHODS: The review adhered to the Arksey and O'Malley process for conducting scoping reviews. The SCOPUS, PUBMED, Web of Science, EMBASE, CINAHL, PsycINFO, ACM Digital Library and IEEE Xplore databases were searched in November, 2020. A two-step selection process identified eligible research. Information was extracted from papers and entered into an Excel coding sheet and summarised. Quality assessments were conducted using the Mixed Methods Appraisal Tool. RESULTS: Twenty-nine studies were included, of which most were of moderate to high quality. Eighteen were observational and 11 were experimental. Twenty-four used robots, four used computer agents and one study used both. The majority of results showed that robots or computer agents positively impacted at least one loneliness outcome measure. Some unintended negative consequences on social outcomes were reported, such as sadness when the robot was removed. Overall, the interventions helped to combat loneliness by acting as a direct companion (69%), a catalyst for social interaction (41%), facilitating remote communication with others (10%) and reminding users of upcoming social engagements (3%). CONCLUSION: Evidence to date suggests that robots can help combat loneliness in older adults, but there is insufficient research on computer agents. Common strategies for reducing loneliness include direct companionship and enabling social interactions. Future research could investigate other strategies used in human interventions (eg, addressing maladaptive social cognition and improving social skills), and the effects of design features on efficacy. It is recommended that more robust experimental and mixed methods research be conducted, using a combination of validated self-report, observational, and interview measures of loneliness.

Analyzing Self-Efficacy and Summary Feedback in Automated Social Skills Training.

Goal: Although automated social skills training has been proposed to enhance human social skills, the following two aspects have not been adequately explored: what types of feedback are effective from virtual agents and the extent to which such systems enhance users' social self-efficacy. Methods: We developed an automated social skills trainer+ that follows human-based social skills training processes and implemented two types of feedback: 1) a summary of the displayed feedback and 2) feedback based on the results of their previous training. Using our developed system, we measured social self-efficacy, feedback evaluations, and the third-party ratings of participants between pre- and post-training as well as their social responsiveness scales. Results: Self-efficacy is significantly correlated to the social responsiveness scale (r = -0.72) and can be improved with our system (mean improvement of 0.68, p < 0.05). The participants highly rated the feedback that was compared to their past training (14 out of 16, p < 0.05) more than the cases without it and the displayed summary feedback (11 out of 16, p = 0.21) more than the verbal comments. Conclusions: Our system effectively summarized user feedback in terms of user self-efficacy and third-party ratings.

A Novel Virtual Coaching System Based on Personalized Clinical Pathways for Rehabilitation of Older Adults-Requirements and Implementation Plan of the vCare Project.

Home-based rehabilitation after an acute episode or following an exacerbation of a chronic disease is often problematic with a clear lack of continuity of care between hospital and home care. Secondary prevention is an essential element of long-term rehabilitation where strategies oriented toward risk reduction, treatment adherence, and optimization of quality of life need to be applied. Frail and sometimes isolated, the patient fails to adhere to the proposed post-discharge clinical pathway due to lack of appropriate clinical, emotional, and informational support. Providing a suitable rehabilitation after an acute episode or a chronic disease is a major issue, as it helps people to live independently and enhance their quality of life. However, as the rehabilitation period usually lasts some months, the continuity of care is often interrupted in the transition from hospital to home. Virtual coaches could help these patients to engage in a personalized rehabilitation program that complies with age-related conditions. These coaches could be a key technology for empowering patients toward increasing their adherence to the care plan and to improve their secondary prevention measures. In this paper, we are presenting a novel virtual coaching system that will address these challenges by combining recent technological advances with clinical pathways, based on joint research and validation activities from researchers from the medical and information and communication technology (ICT) domains.

Assessment of users' acceptability of a mobile-based embodied conversational agent for the prevention and detection of suicidal behaviour.

The use of embodied conversational agents in mental health has increased in the last years. Several studies exist describing the benefits and advantages of this technology as a complement to psychotherapeutic interventions for the prevention and treatment of depression, anxiety, or post-traumatic stress disorder, to name a few. A small number of these works implement capabilities in the virtual agent focused on the detection and prevention of suicidality risks. The work presented in this paper describes the development of an embodied conversational agent used as the main interface in HelPath, a mobile-based application addressed to individuals detected with any of the suicidal behaviours: ideation, planning or attempt. The main objective of HelPath is to continuously collect user's information that, complemented with data from the electronic health record, supports the identification of risks associated with suicidality. Through the virtual agent, the users also receive information and suggestions based on cognitive behaviour therapy that would help them to maintain a healthy condition. The paper also presents the execution of an exploratory pilot to assess the acceptability, perception and adherence of users towards the virtual agent. The obtained results are presented and discussed, and some actions for further improvement of the embodied conversational agent are also identified.

Managing an Agent's Self-Presentational Strategies During an Interaction.

In this paper we present a computational model for managing the impressions of warmth and competence (the two fundamental dimensions of social cognition) of an Embodied Conversational Agent (ECA) while interacting with a human. The ECA can choose among four different self-presentational strategies eliciting different impressions of warmth and/or competence in the user, through its verbal and non-verbal behavior. The choice of the non-verbal behaviors displayed by the ECA relies on our previous studies. In our first study, we annotated videos of human-human natural interactions of an expert on a given topic talking to a novice, in order to find associations between the warmth and competence elicited by the expert's non-verbal behaviors (such as type of gestures, arms rest poses, smiling). In a second study, we investigated whether the most relevant non-verbal cues found in the previous study were perceived in the same way when displayed by an ECA. The computational learning model presented in this paper aims to learn in real-time the best strategy (i.e., the degree of warmth and/or competence to display) for the ECA, that is, the one which maximizes user's engagement during the interaction. We also present an evaluation study, aiming to investigate our model in a real context. In the experimental scenario, the ECA plays the role of a museum guide introducing an exposition about video games. We collected data from 75 visitors of a science museum. The ECA was displayed in human dimension on a big screen in front of the participant, with a Kinect on the top. During the interaction, the ECA could adopt one of 4 self-presentational strategies during the whole interaction, or it could select one strategy randomly for each speaking turn, or it could use a reinforcement learning algorithm to choose the strategy having the highest reward (i.e., user's engagement) after each speaking turn.

Development and validation of a virtual agent to screen tobacco and alcohol use disorders.

BACKGROUND: Substance use disorders are under-detected and not systematically diagnosed or screened for by primary care. In this study, we present the acceptability and validity of an Embodied Conversational Agent (ECA) designed to screen tobacco and alcohol use disorder, in individuals who did not seek medical help for these disorders. METHODS: Individuals were included from June 2016 to May 2017 in the Outpatient Sleep Clinic of the University Hospital of Bordeaux. DSM-5 diagnoses of tobacco and alcohol use disorders were assessed by human interviewers. The ECA interview integrated items from the Cigarette Dependence Scale-5 (CDS-5) for tobacco use disorder screening, and the "Cut Down, Annoyed, Guilty, Eye-opener" (CAGE) questionnaire for alcohol use disorder screening. Paper version of CDS-5 and CAGE questionnaires and acceptability questionnaire was also self-administered. RESULTS: Of the 139 participants in the study (mean age 43.0 [SD = 13.7] years), 71 were women, and 68 were men. The ECA was well accepted by the patients. Paper self-administered CDS-5 and CAGE scores had a strong agreement with the ECA (p < 0.0001). The Receiver Operating Characteristic (ROC) analysis of the ECA interview showed AUC of 0.97 (95% CI, 0.93-1.0) and 0.84 (95% CI, 0.69-0.98) for CDS-5 and CAGE respectively with p-value <0.0001. CONCLUSIONS: This ECA was acceptable and valid to screen tobacco or alcohol use disorder among patients not requesting treatment for addiction. The ECA could be used in hospitals and potentially in primary care settings to help clinicians to better screen their patients for alcohol and tobacco use disorders.

Automating the Production of Communicative Gestures in Embodied Characters.

In this paper we highlight the different challenges in modeling communicative gestures for Embodied Conversational Agents (ECAs). We describe models whose aim is to capture and understand the specific characteristics of communicative gestures in order to envision how an automatic communicative gesture production mechanism could be built. The work is inspired by research on how human gesture characteristics (e.g., shape of the hand, movement, orientation and timing with respect to the speech) convey meaning. We present approaches to computing where to place a gesture, which shape the gesture takes and how gesture shapes evolve through time. We focus on a particular model based on theoretical frameworks on metaphors and embodied cognition that argue that people can represent, reason about and convey abstract concepts using physical representations and processes, which can be conveyed through physical gestures.

Embodied Conversational Agents for the Detection and Prevention of Suicidal Behaviour: Current Applications and Open Challenges.

Embodied conversational agents (ECAs) are advanced computational interactive interfaces designed with the aim to engage users in the continuous and long-term use of a background application. The advantages and benefits of these agents have been exploited in several e-health systems. One of the medical domains where ECAs are recently applied is to support the detection of symptoms, prevention and treatment of mental health disorders. As ECAs based applications are increasingly used in clinical psychology, and due that one fatal consequence of mental health problems is the commitment of suicide, it is necessary to analyse how current ECAs in this clinical domain support the early detection and prevention of risk situations associated with suicidality. The present work provides and overview of the main features implemented in the ECAs to detect and prevent suicidal behaviours through two scenarios: ECAs acting as virtual counsellors to offer immediate help to individuals in risk; and ECAs acting as virtual patients for learning/training in the identification of suicide behaviours. A literature review was performed to identify relevant studies in this domain during the last decade, describing the main characteristics of the implemented ECAs and how they have been evaluated. A total of six studies were included in the review fulfilling the defined search criteria. Most of the experimental studies indicate promising results, though these types of ECAs are not yet commonly used in routine practice. The identification of some open challenges for the further development of ECAs within this domain is also discussed.

Perceived organizational affiliation and its effects on patient trust: Role modeling with embodied conversational agents.

OBJECTIVE: Verbal and non-verbal behaviors, which are known as "relational contextualization cues", relay information about relationships and how they are structured. We developed a computer-simulated provider conducting an informed consent process for clinical research to investigate the effects of a provider's alignment of interests with a patient, the research team, or a neutral party on patient trust in the provider. METHODS: Participants (N=43) interacted with a simulated provider for a research informed consent process in a three-arm, counterbalanced, within-subjects experiment. Participants reported their trust in the simulated provider after each treatment. RESULTS: Participants successfully recognized the alignment manipulation, and perceived the patient-aligned provider as more trustworthy than the other providers. Participants were also more satisfied with the patient-aligned provider, liked this provider more, expressed more desire to continue working with this provider, and stated that they were significantly more likely to sign the consent form after interacting with this provider compared to the other two. CONCLUSION: Relational contextualization that aligns with the patient increases trust, satisfaction, and willingness to enroll in the context of research informed consent. PRACTICE IMPLICATIONS: Health providers should align themselves with patients' interests.

Virtual training and coaching of health behavior: example from mindfulness meditation training.

OBJECTIVE: Computer-based virtual coaches are increasingly being explored for patient education, counseling, and health behavior training and coaching. The objective of this research was to develop and evaluate a Virtual Mindfulness Coach for training and coaching in mindfulness meditation. METHODS: The coach was implemented as an embodied conversational character, providing mindfulness training and coaching via mixed initiative, text-based, natural language dialog with the student, and emphasizing affect-adaptive interaction. (The term 'mixed initiative dialog' refers to a human-machine dialog where either can initiate a conversation or a change in the conversation topic.) RESULTS: Findings from a pilot evaluation study indicate that the coach-based training is more effective in helping students establish a regular practice than self-administered training using written and audio materials. The coached group also appeared to be in more advanced stages of change in terms of the transtheoretical model, and have a higher sense of self-efficacy regarding establishment of a regular mindfulness practice. CONCLUSION: These results suggest that virtual coach-based training of mindfulness is both feasible, and potentially more effective, than a self-administered program. Of particular interest is the identification of the specific coach features that contribute to its effectiveness. PRACTICE IMPLICATIONS: Virtual coaches could provide easily accessible and cost-effective customized training for a range of health behaviors. The affect-adaptive aspect of these coaches is particularly relevant for helping patients establish long-term behavior changes.