General Characteristics and Design Taxonomy of Chatbots for COVID-19: Systematic Review.

BACKGROUND: A conversational agent powered by artificial intelligence, commonly known as a chatbot, is one of the most recent innovations used to provide information and services during the COVID-19 pandemic. However, the multitude of conversational agents explicitly designed during the COVID-19 pandemic calls for characterization and analysis using rigorous technological frameworks and extensive systematic reviews. OBJECTIVE: This study aims to describe the general characteristics of COVID-19 chatbots and examine their system designs using a modified adapted design taxonomy framework. METHODS: We conducted a systematic review of the general characteristics and design taxonomy of COVID-19 chatbots, with 56 studies included in the final analysis. This review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines to select papers published between March 2020 and April 2022 from various databases and search engines. RESULTS: Results showed that most studies on COVID-19 chatbot design and development worldwide are implemented in Asia and Europe. Most chatbots are also accessible on websites, internet messaging apps, and Android devices. The COVID-19 chatbots are further classified according to their temporal profiles, appearance, intelligence, interaction, and context for system design trends. From the temporal profile perspective, almost half of the COVID-19 chatbots interact with users for several weeks for >1 time and can remember information from previous user interactions. From the appearance perspective, most COVID-19 chatbots assume the expert role, are task oriented, and have no visual or avatar representation. From the intelligence perspective, almost half of the COVID-19 chatbots are artificially intelligent and can respond to textual inputs and a set of rules. In addition, more than half of these chatbots operate on a structured flow and do not portray any socioemotional behavior. Most chatbots can also process external data and broadcast resources. Regarding their interaction with users, most COVID-19 chatbots are adaptive, can communicate through text, can react to user input, are not gamified, and do not require additional human support. From the context perspective, all COVID-19 chatbots are goal oriented, although most fall under the health care application domain and are designed to provide information to the user. CONCLUSIONS: The conceptualization, development, implementation, and use of COVID-19 chatbots emerged to mitigate the effects of a global pandemic in societies worldwide. This study summarized the current system design trends of COVID-19 chatbots based on 5 design perspectives, which may help developers conveniently choose a future-proof chatbot archetype that will meet the needs of the public in the face of growing demand for a better pandemic response.

[Virtual reality in teaching using mentally ill patient avatars]. / Virtuelle Realität in der Lehre mit psychisch kranken Patientenavataren.

BACKGROUND: Medical interaction and exploration techniques are the most important tools that medical students have to acquire in the subject of psychiatry and psychotherapy. The new digital technologies currently available, such as virtual reality (VR), as important supplements can contribute to a significant improvement in the teaching of psychiatric-psychopathological learning content as well as, in particular, the technique of ascertaining the psychiatric history and diagnosis. OBJECTIVE: Evaluation of the Bochum Avatar Exploration Project (AVEX) as part of the curricular course in medical studies at the Ruhr University Bochum for its possibilities to convey learning content and techniques of anamnesis and diagnosis in the subject of psychiatry and psychotherapy. METHODS: In AVEX, a total of 87 medical students in the clinical study section have so far been able to enter into a dialogue with "mentally ill" avatars and gain experience with VR technology as a learning and teaching method in the subject of psychiatry and psychotherapy. RESULTS: Despite the limited possibilities for interaction with the digital avatars, it is possible to achieve a substantial transfer of learning content in psychiatry; however, the students must be well supported by the lecturers. CONCLUSION: The AVEX project already shows promising possibilities for supplementing the teaching of medical students, even if the fit of questions and replies in dialogue with the virtual avatars still needs to be improved. As advances in the linguistic communication of emotions and the visual effects of the avatar representation can be predicted, the significance of this technology will continue to increase.

The (Mis)Information Game: A social media simulator.

Given the potential negative impact reliance on misinformation can have, substantial effort has gone into understanding the factors that influence misinformation belief and propagation. However, despite the rise of social media often being cited as a fundamental driver of misinformation exposure and false beliefs, how people process misinformation on social media platforms has been under-investigated. This is partially due to a lack of adaptable and ecologically valid social media testing paradigms, resulting in an over-reliance on survey software and questionnaire-based measures. To provide researchers with a flexible tool to investigate the processing and sharing of misinformation on social media, this paper presents The Misinformation Game-an easily adaptable, open-source online testing platform that simulates key characteristics of social media. Researchers can customize posts (e.g., headlines, images), source information (e.g., handles, avatars, credibility), and engagement information (e.g., a post's number of likes and dislikes). The platform allows a range of response options for participants (like, share, dislike, flag) and supports comments. The simulator can also present posts on individual pages or in a scrollable feed, and can provide customized dynamic feedback to participants via changes to their follower count and credibility score, based on how they interact with each post. Notably, no specific programming skills are required to create studies using the simulator. Here, we outline the key features of the simulator and provide a non-technical guide for use by researchers. We also present results from two validation studies. All the source code and instructions are freely available online at https://misinfogame.com .

How storytelling through the use of avatars could help Fukushima evacuees express themselves: digital archive for people in the diaspora.

The Memorytalk online platform allows users to upload animated human stories. They may also choose to display related photos/characters and construct scenarios. Memorytalk may thus constitute a useful collaborative format for survivors of the Great East Japan Earthquake, especially evacuees from the town of Namie, who are now relocated in different areas across Japan, particularly as they are hesitant to talk about where they are from. This study examined various narrated animations created by Namie high school students within the Memorytalk platform, specifically for the purpose of investigating their storytelling methods based on Erving Goffman's role-distance theory. Memorytalk not only allows avatars that provide users with anonymity when expressing a wide range of memories, it also creates (1) opportunities for face-to-face workshops in the context of computer-mediated communication and (2) the therapeutic benefits of being humorous. In sum, the platform may provide users with a form of role distancing in which people in the diaspora can more freely express themselves and also store valuable local memories.

Extended Reality Telemedicine Collaboration System Using Patient Avatar Based on 3D Body Pose Estimation.

Recent advances in extended reality (XR) technology have opened the possibility of significantly improving telemedicine systems. This is primarily achieved by transferring 3D information about patient state, which is utilized to create more immersive experiences on VR/AR headsets. In this paper, we propose an XR-based telemedicine collaboration system in which the patient is represented as a 3D avatar in an XR space shared by local and remote clinicians. The proposed system consists of an AR client application running on Microsoft HoloLens 2 used by a local clinician, a VR client application running on the HTC vive Pro used by a remote clinician, and a backend part running on the server. The patient is captured by a camera on the AR side, and the 3D body pose estimation is performed on frames from this camera stream to form a 3D patient avatar. Additionally, the AR and VR sides can interact with the patient avatar via virtual hands, and annotations can be performed on a 3D model. The main contribution of our work is the use of 3D body pose estimation for the creation of a 3D patient avatar. In this way, 3D body reconstruction using depth cameras is avoided, which reduces system complexity and hardware and network resources. Another contribution is the novel architecture of the proposed system, where audio and video streaming are realized using WebRTC protocol. The performance evaluation showed that the proposed system ensures high frame rates for both AR and VR client applications, while the processing latency remains at an acceptable level.

Digital Avatars and Personalized Voices-How AI Is Helping to Restore Speech to Patients.

In this Medical News article, Edward Chang, MD, chair of the department of neurological surgery at the University of California, San Francisco Weill Institute for Neurosciences joins JAMA Editor in Chief Kirsten Bibbins-Domingo, PhD, MD, MAS, to discuss the potential for AI to revolutionize communication for those unable to speak due to aphasia.

Embodying a self-avatar with a larger leg: its impacts on motor control and dynamic stability.

Several studies have shown that users of immersive virtual reality can feel high levels of embodiment in self-avatars that have different morphological proportions than those of their actual bodies. Deformed and unrealistic morphological modifications are accepted by embodied users, underlying the adaptability of one's mental map of their body (body schema) in response to incoming sensory feedback. Before initiating a motor action, the brain uses the body schema to plan and sequence the necessary movements. Therefore, embodiment in a self-avatar with a different morphology, such as one with deformed proportions, could lead to changes in motor planning and execution. In this study, we aimed to measure the effects on movement planning and execution of embodying a self-avatar with an enlarged lower leg on one side. Thirty participants embodied an avatar without any deformations, and with an enlarged dominant or non-dominant leg, in randomized order. Two different levels of embodiment were induced, using synchronous or asynchronous visuotactile stimuli. In each condition, participants performed a gait initiation task. Their center of mass and center of pressure were measured, and the margin of stability (MoS) was computed from these values. Their perceived level of embodiment was also measured, using a validated questionnaire. Results show no significant changes on the biomechenical variables related to dynamic stability. Embodiment scores decreased with asynchronous stimuli, without impacting the measures related to stability. The body schema may not have been impacted by the larger virtual leg. However, deforming the self-avatar's morphology could have important implications when addressing individuals with impaired physical mobility by subtly influencing action execution during a rehabilitation protocol.

What digital health technology types are used in mental health prevention and intervention? Review of systematic reviews for systematization of technologies.

Digital health technology has been widely applied to mental health interventions worldwide. Using digital phenotyping to identify an individual's mental health status has become particularly important. However, many technologies other than digital phenotyping are expected to become more prevalent in the future. The systematization of these technologies is necessary to accurately identify trends in mental health interventions. However, no consensus on the technical classification of digital health technologies for mental health interventions has emerged. Thus, we conducted a review of systematic review articles on the application of digital health technologies in mental health while attempting to systematize the technology using the Delphi method. To identify technologies used in digital phenotyping and other digital technologies, we included 4 systematic review articles that met the inclusion criteria, and an additional 8 review articles, using a snowballing approach, were incorporated into the comprehensive review. Based on the review results, experts from various disciplines participated in the Delphi process and agreed on the following 11 technical categories for mental health interventions: heart rate estimation, exercise or physical activity, sleep estimation, contactless heart rate/pulse wave estimation, voice and emotion analysis, self-care/cognitive behavioral therapy/mindfulness, dietary management, psychological safety, communication robots, avatar/metaverse devices, and brain wave devices. The categories we defined intentionally included technologies that are expected to become widely used in the future. Therefore, we believe these 11 categories are socially implementable and useful for mental health interventions.

Exploring the Influence of Virtual Avatar Heads in Mixed Reality on Social Presence, Performance and User Experience in Collaborative Tasks.

In Mixed Reality (MR), users' heads are largely (if not completely) occluded by the MR Head-Mounted Display (HMD) they are wearing. As a consequence, one cannot see their facial expressions and other communication cues when interacting locally. In this paper, we investigate how displaying virtual avatars' heads on-top of the (HMD-occluded) heads of participants in a Video See-Through (VST) Mixed Reality local collaborative task could improve their collaboration as well as social presence. We hypothesized that virtual heads would convey more communicative cues (such as eye direction or facial expressions) hidden by the MR HMDs and lead to better collaboration and social presence. To do so, we conducted a between-subject study ($\mathrm{n}=88$) with two independent variables: the type of avatar (CartoonAvatar/RealisticAvatar/NoAvatar) and the level of facial expressions provided (HighExpr/LowExpr). The experiment involved two dyadic communication tasks: (i) the "20-question" game where one participant asks questions to guess a hidden word known by the other participant and (ii) a urban planning problem where participants have to solve a puzzle by collaborating. Each pair of participants performed both tasks using a specific type of avatar and facial animation. Our results indicate that while adding an avatar's head does not necessarily improve social presence, the amount of facial expressions provided through the social interaction does have an impact. Moreover, participants rated their performance higher when observing a realistic avatar but rated the cartoon avatars as less uncanny. Taken together, our results contribute to a better understanding of the role of partial avatars in local MR collaboration and pave the way for further research exploring collaboration in different scenarios, with different avatar types or MR setups.

In Case of Doubt, One Follows One's Self: The Implicit Guidance of the Embodied Self-Avatar.

The sense of embodiment in virtual reality (VR) is commonly understood as the subjective experience that one's physical body is substituted by a virtual counterpart, and is typically achieved when the avatar's body, seen from a first-person view, moves like one's physical body. Embodiment can also be experienced in other circumstances (e.g., in third-person view) or with imprecise or distorted visuo-motor coupling. It was moreover observed, in various cases of small or progressive temporal and spatial manipulations of avatars' movements, that participants may spontaneously follow the movement shown by the avatar. The present work investigates whether, in some specific contexts, participants would follow what their avatar does even when large movement discrepancies occur, thereby extending the scope of understanding of the self-avatar follower effect beyond subtle changes of motion or speed manipulations. We conducted an experimental study in which we introduced uncertainty about which movement to perform at specific times and analyzed participants' movements and subjective feedback after their avatar showed them an incorrect movement. Results show that, when in doubt, participants were influenced by their avatar's movements, leading them to perform that particular error twice more often than normal. Importantly, results of the embodiment score indicate that participants experienced a dissociation with their avatar at those times. Overall, these observations not only demonstrate the possibility of provoking situations in which participants follow the guidance of their avatar for large motor distortions, despite their awareness about the avatar movement disruption and on the possible influence it had on their choice, and, importantly, exemplify how the cognitive mechanism of embodiment is deeply rooted in the necessity of having a body.

User Training With Error Augmentation for sEMG-Based Gesture Classification.

We designed and tested a system for real-time control of a user interface by extracting surface electromyographic (sEMG) activity from eight electrodes in a wristband configuration. sEMG data were streamed into a machine-learning algorithm that classified hand gestures in real-time. After an initial model calibration, participants were presented with one of three types of feedback during a human-learning stage: veridical feedback, in which predicted probabilities from the gesture classification algorithm were displayed without alteration; modified feedback, in which we applied a hidden augmentation of error to these probabilities; and no feedback. User performance was then evaluated in a series of minigames, in which subjects were required to use eight gestures to manipulate their game avatar to complete a task. Experimental results indicated that relative to the baseline, the modified feedback condition led to significantly improved accuracy. Class separation also improved, though this trend was not significant. These findings suggest that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications.

The effect of gender on emotional reactions and perceptions when individuals meet themselves in immersive virtual reality.

Immersive virtual reality-based training and research are becoming more and more popular and are in continuous development. For instance, it is now possible to be trained by one's virtual self (i.e., doppelganger), meaning that a trainee can participate in a training program in which the trainer resembles the trainee. While past research involving doppelgangers showed promising results, findings revealed gender effects such that doppelganger-based training seems to be beneficial only for male trainees. In the present research, we contribute to this literature by investigating the emotional reactions and perceptions that people have when they meet a virtual human in immersive virtual reality. Specifically, we assess the extent to which the appearance of the virtual human (doppelganger vs. unknown avatar) and an individual's gender influence these reactions and perceptions. We found that males felt more positive emotions toward their doppelganger than toward the unknown avatar and that females perceived their doppelganger as less competent and warm than the unknown avatar. Our findings have important practical implications in terms of virtual reality-based training and research design such that the use of a doppelganger (unknown avatar) might be recommended in a training program or research setting involving men (women).

Customizing the human-avatar mapping based on EEG error related potentials.

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

Sexual Violence in Virtual Reality: A Scoping Review.

BACKGROUND: One in four Americans report experiencing harassment online via social media and interactive gaming, which includes physical threats, stalking, sexual harassment, and sustained harassment. OBJECTIVE: The aim of this study was to gain understanding of the state of the science surrounding young adults and sexual violence/harassment harms in virtual reality (VR) as well as possible uses of VR to heal and intervene. METHODS: A scoping review was conducted in early 2023 using the Ovid Synthesis Clinical Evidence Manager and the MEDLINE database. Forty-seven articles met inclusion criteria. RESULTS: Our review found a growing body of evidence exploring incidents, effects, possible predictors, and initial strategies to prevent sexual violence in VR and to use the modality to positively intervene. Limited research addresses the effects of harms incurred in VR on (re)traumatization of survivors as well as the development and testing of VR tools used to educate, deliver bystander interventions, transform biases and perceptions via embodiment, and promote healing among survivors. CONCLUSION: Research addressing sexual violence in VR is needed and should build on the existing peripheral science on gaming and social media environments. Forensic nursing is well positioned to advance strategies of health and safety in VR, just as in the physical world. Incorporating forensic nursing avatars in VR and deploying diverse resources targeted for college-age young adults to prevent harms in VR should be explored safely and ethically. Forensic nurses are also positioned to assess for VR-related harms among patients and to work with private and government sectors to influence regulations and policies.

The Utilitarian Virtual Self - Using Embodied Personalized Avatars to Investigate Moral Decision-Making in Semi-Autonomous Vehicle Dilemmas.

Embodied personalized avatars are a promising new tool to investigate moral decision-making by transposing the user into the "middle of the action" in moral dilemmas. Here, we tested whether avatar personalization and motor control could impact moral decision-making, physiological reactions and reaction times, as well as embodiment, presence and avatar perception. Seventeen participants, who had their personalized avatars created in a previous study, took part in a range of incongruent (i.e., harmful action led to better overall outcomes) and congruent (i.e., harmful action led to trivial outcomes) moral dilemmas as the drivers of a semi-autonomous car. They embodied four different avatars (counterbalanced - personalized motor control, personalized no motor control, generic motor control, generic no motor control). Overall, participants took a utilitarian approach by performing harmful actions only to maximize outcomes. We found increased physiological arousal (SCRs and heart rate) for personalized avatars compared to generic avatars, and increased SCRs in motor control conditions compared to no motor control. Participants had slower reaction times when they had motor control over their avatars, possibly hinting at more elaborate decision-making processes. Presence was also higher in motor control compared to no motor control conditions. Embodiment ratings were higher for personalized avatars, and generally, personalization and motor control were perceptually positive features. These findings highlight the utility of personalized avatars and open up a range of future research possibilities that could benefit from the affordances of this technology and simulate, more closely than ever, real-life action.

Stepping into the Right Shoes: The Effects of User-Matched Avatar Ethnicity and Gender on Sense of Embodiment in Virtual Reality.

In many consumer virtual reality (VR) applications, users embody predefined characters that offer minimal customization options, frequently emphasizing storytelling over user choice. We explore whether matching a user's physical characteristics, specifically ethnicity and gender, with their virtual self-avatar affects their sense of embodiment in VR. We conducted a $2\times 2$ within-subjects experiment ($\mathrm{n}=32$) with a diverse user population to explore the impact of matching or not matching a user's self-avatar to their ethnicity and gender on their sense of embodiment. Our results indicate that matching the ethnicity of the user and their self-avatar significantly enhances sense of embodiment regardless of gender, extending across various aspects, including appearance, response, and ownership. We also found that matching gender significantly enhanced ownership, suggesting that this aspect is influenced by matching both ethnicity and gender. Interestingly, we found that matching ethnicity specifically affects self-location while matching gender specifically affects one's body ownership.

Impact of using virtual avatars in educational videos on user experience.

Popularization of knowledge is of considerable importance and necessity, and traditional knowledge popularization activities suffer from high cost and low acceptance, which affect their effectiveness and coverage. Applying virtual avatars to educational videos may be an effective way to solve the problem. This study investigates the impact of applying virtual avatars to educational videos on user experience. Constructed a model of the impact of user experience on educational videos with virtual avatars, collected data from the target population, and analyzed it empirically. The video quality and virtual avatar expressiveness dimensions of the influencing factors have a significant positive effect on the learning effect, emotional experience and user engagement dimensions of user experience; the content quality dimension of the influencing factors has a significant negative effect on the three dimensions of user experience. The video quality and virtual avatar expressiveness dimensions of the influencing factors have a significant positive effect on the learning effect, emotional experience and user engagement dimensions of user experience; the content quality dimension of the influencing factors has a significant negative effect on the three dimensions of user experience.

Gender Moderates the Relationship Between Avatar Customization and Enjoyment in Popular Video Games.

Using online survey methods (n = 153), this study investigates whether exposure to different levels of customization in games will increase the experiences of enjoyment, both directly and indirectly, through the experience of avatar embodiment, as well as whether these relationships are influenced by gender identity. Results indicate a positive relationship between level of exposure to customization and enjoyment for women-identified participants. These outcomes may provide insight into gendered preference in gaming, potentially providing an avenue by which to make games more inclusive to women gamers.

Virtually simulated interpersonal touch negatively affects perceived closeness and social affiliation to an avatar partner.

Interpersonal touch is an essential component of human non-verbal communication, facilitating social affiliation and bonding. With the widespread use of digital interfaces and online platforms in all realms of human interactions, there are fewer opportunities for communicating through touch. Popular online platforms that virtually simulate human interactions rely primarily on visual and auditory modalities, providing limited or no capacity for the exchange of tactile cues. Previous studies of virtual interactions have explored the simulation of social touch using haptic devices, but little is known about how the visual representation of interpersonal touch is perceived and integrated into a virtual social experience. In two studies we examined how the exchange of virtual touch mediated by simulated 3-dimensional human characters, or avatars, within an online virtual environment influenced affiliation towards an unfamiliar interaction partner. Surprisingly, the exchange of virtual touch negatively affected the perceived closeness and affiliation to the partner and the social evaluation of the interaction but did not affect the level of physiological arousal during the interaction. These results indicate that the visual representation of social touch is sufficient to virtually communicate touch-related cues that impact social affiliation, but the influence of touch may be dependent on the interaction context.