Frankenstein's Monster in the Metaverse: User Interaction With Customized Virtual Agents.

Enabled by the latest achievements in artificial intelligence (AI), computer graphics as well as virtual, augmented, and mixed reality (VR/AR/MR), virtual agents are increasingly resembling humans in both their appearance and intelligent behavior. This results in enormous potential for agents to support users in their daily lives, for example in customer service, healthcare, education or the envisioned all-encompassing metaverse. Today's technology would allow users to customize their conversation partners in the metaverse - as opposed to reality - according to their preferences, potentially improving the user experience. On the other hand, there is little research on how reshaping the head of a communication partner might affect the immediate interaction with them. In this paper, we investigate the user requirements for and the effects of agent customization. In a two-stage user study (N = 30), we collected both self-reported evaluations (e.g., intrinsic motivation) and interaction metrics (e.g., interaction duration and number of tried out items) for the process of agent customization itself as well as data on how users perceived the subsequent human-agent interaction in VR. Our results indicate that users only wish to have full customization for agents in their personal social circle, while for general services, a selection or even a definite assignment of pre-configured agents is sufficient. When customization is offered, attributes such as gender, clothing or hair are subjectively more relevant to users than facial features such as skin or eye color. Although the customization of human interaction partners is beyond our control, customization of virtual agents significantly increases perceived social presence as well as rapport and trust. Further findings on user motivation and agent diversity are discussed in the paper.

SceneFusion: Room-Scale Environmental Fusion for Efficient Traveling Between Separate Virtual Environments.

Traveling between scenes has become a major requirement for navigation in numerous virtual reality (VR) social platforms and game applications, allowing users to efficiently explore multiple virtual environments (VEs). To facilitate scene transition, prevalent techniques such as instant teleportation and virtual portals have been extensively adopted. However, these techniques exhibit limitations when there is a need for frequent travel between separate VEs, particularly within indoor environments, resulting in low efficiency. In this paper, we first analyze the design rationale for a novel navigation method supporting efficient travel between virtual indoor scenes. Based on the analysis, we introduce the SceneFusion technique that fuses separate virtual rooms into an integrated environment. SceneFusion enables users to perceive rich visual information from both rooms simultaneously, achieving high visual continuity and spatial awareness. While existing teleportation techniques passively transport users, SceneFusion allows users to actively access the fused environment using short-range locomotion techniques. User experiments confirmed that SceneFusion outperforms instant teleportation and virtual portal techniques in terms of efficiency, workload, and preference for both single-user exploration and multi-user collaboration tasks in separate VEs. Thus, SceneFusion presents an effective solution for seamless traveling between virtual indoor scenes.

Adding virtual plants leads to higher cognitive performance and psychological well-being in virtual reality.

Previous research has shown the positive effects of exposure to real and virtual nature. To investigate how such benefits might generalize to ever-more-prevalent virtual workplaces, we examined the effects of the absence or presence of virtual plants in an office environment in Virtual Reality (VR) on users' cognitive performance and psychological well-being. The results of our user study with 39 participants show that in the presence of virtual plants, participants performed significantly better in both short-term memory and creativity tasks. Furthermore, they reported higher psychological well-being scores, including positive affect and attentive coping, whilst reporting lower feelings of anger and aggression after exposure to virtual plants in VR. The virtual office with plants was also perceived as more restorative and induced a higher sense of presence. Overall, these results highlight how the presence of virtual plants in VR can have positive influences on users, and therefore, constitute important design considerations when developing future working and learning spaces.

Effects of exposure to immersive computer-generated virtual nature and control environments on affect and cognition.

Previous research has shown that exposure to immersive virtual nature environments is able to induce positive affective and physiological effects. However, research on the effects on cognitive performance is scarce. Additionally, the effects of virtual nature exposure compared to a virtual control environment with a comparable amount of virtual objects have not been examined so far. Therefore, we conducted an experiment with 27 participants to study the psychological effects of such exposure. The virtual nature consisted of a 3D model of a typical forest environment, whereas the control environment was an abstract replication of the virtual forest environment. In both environments, a virtual wooden cart was used to transport the users from the start to the end of the virtual road. The typical background noise of moving such a cart was integrated into both environments as well. In addition, the virtual nature environment included typical forest sounds in the background, whereas the control condition did not have such background sounds. Both environments were compared with regard to their effects on cognitive performance (using trail making tests (TMTA, TMTB, and TMTB-A) as well as digit span forward and digit span backward tests), perceived restorativeness, mood, stress, sense of presence, and simulator sickness. The results showed that in comparison to the control environment, exposure to the virtual nature resulted in significantly higher cognitive performance, higher perceived restorativeness, higher positive affect, higher sense of presence, lower perceived stress, and lower simulator sickness.

Enriched environments enhance the development of explicit memory in an incidental learning task.

Learning, rendered in an implicit (unconscious) or explicit (conscious) way, is a crucial part of our daily life. Different factors, like attention or motivation, influence the transformation from implicit to explicit memory. Via virtual reality a lively and engaging surrounding can be created, whereby motivational processes are assumed to be a vital part of the transition from implicit to explicit memory. In the present study, we tested the impact of an enriched virtual reality compared to two conventional, non-enriched 2D-computer-screen based tasks on implicit to explicit memory transformation, using an audio-visual sequential association task. We hypothesized, that the immersive nature of the VR surrounding enhances the transfer from implicit to explicit memory. Notably, the overall amount of learned sequence pairs were not significantly different between experimental groups, but the degree of awareness was affected by the different settings. However, we observed an increased level of explicitly remembered pairs within the VR group compared to two screen-based groups. This finding clearly demonstrates that a near-natural experimental setting affects the transformation process from implicit to explicit memory.

Transferable Virtual-Physical Environmental Alignment with Redirected Walking.

Several advanced redirected walking techniques have been proposed in recent years to improve natural walking in virtual environments. One active and important research challenge of redirected walking focuses on the alignment of virtual and physical environments by redirection gains. If both environments are aligned, physical objects appear at the same positions as their virtual counterparts. When a user arrives at such a virtual object, she can touch the corresponding physical object providing passive haptic feedback. When multiple transferable virtual or physical target positions exist, the alignment can exploit multiple options, but the process requires more complicated solutions. In this paper, we study the problem of virtual-physical environmental alignment at multiple transferable target positions, and introduce a novel reinforcement learning-based redirected walking method. We design a novel comprehensive reward function that dynamically determines virtual-physical target matching and updates virtual target weights for reward computation. We evaluate our method through various simulated experiments as well as real user tests. The results show that our method obtains less physical distance error for environmental alignment and requires fewer resets than state-of-the-art techniques.

A Review of the Potential of Virtual Walking Techniques for Gait Rehabilitation.

Virtual reality (VR) technology has emerged as a promising tool for studying and rehabilitating gait disturbances in different cohorts of patients (such as Parkinson's disease, post-stroke, or other neurological disorders) as it allows patients to be engaged in an immersive and artificial environment, which can be designed to address the particular needs of each individual. This review demonstrates the state of the art in applications of virtual walking techniques and related technologies for gait therapy and rehabilitation of people with movement disorders makes recommendations for future research and discusses the use of VR in the clinic. However, the potential for using these techniques in gait rehabilitation is to provide a more personalized approach by simulate the experience of natural walking, while patients with neurological disorders are maintained localized in the real world. The goal of our work is to investigate how the human nervous system controls movement in health and neurodegenerative disease.

Capturing Workplace Gossip as Dynamic Conversational Events: First Insights From Care Team Meetings.

Even though gossip is a ubiquitous organizational behavior that fulfils important social functions (e.g., social bonding or emotion venting), little is known about how workplace gossip and its functions unfold in situ. To explore the dynamic nature and social embeddedness of workplace gossip, we develop a behavioral annotation system that captures the manifold characteristics of verbal gossip behavior, including its valence and underlying functions. We apply this system to eight elderly care team meetings audio- and videotaped in the field, yielding a sample of N = 4,804 annotated behaviors. On this empirical basis, we provide first insights into the different facets and functions of workplace gossip in real-life team interactions. By means of lag sequential analysis, we quantify gossip patterns that point to the temporal and structural embeddedness of different types of workplace gossip expressions. Though exploratory, these findings help establish workplace gossip as a dynamic conversational event. We discuss future interdisciplinary research collaborations that behavioral observation approaches offer.

Effects of exposure to immersive videos and photo slideshows of forest and urban environments.

A large number of studies have demonstrated the benefits of natural environments on people's health and well-being. For people who have limited access to nature (e.g., elderly in nursing homes, hospital patients, or jail inmates), virtual representations may provide an alternative to benefit from the illusion of a natural environment. For this purpose and in most previous studies, conventional photos of nature have been used. Immersive virtual reality (VR) environments, however, can induce a higher sense of presence compared to conventional photos. Whether this higher sense of presence leads to increased positive impacts of virtual nature exposure is the main research question of this study. Therefore, we compared exposure to a forest and an urban virtual environment in terms of their respective impact on mood, stress, physiological reactions, and cognition. The environments were presented via a head-mounted display as (1) conventional photo slideshows or (2) 360[Formula: see text] videos. The results show that the forest environment had a positive effect on cognition and the urban environment disturbed mood regardless of the mode of presentation. In addition, photos of either urban or forest environment were both more effective in reducing physiological arousal compared to immersive 360[Formula: see text] videos.

A Comparison of Eye Tracking Latencies Among Several Commercial Head-Mounted Displays.

A number of virtual reality head-mounted displays (HMDs) with integrated eye trackers have recently become commercially available. If their eye tracking latency is low and reliable enough for gaze-contingent rendering, this may open up many interesting opportunities for researchers. We measured eye tracking latencies for the Fove-0, the Varjo VR-1, and the High Tech Computer Corporation (HTC) Vive Pro Eye using simultaneous electrooculography measurements. We determined the time from the occurrence of an eye position change to its availability as a data sample from the eye tracker (delay) and the time from an eye position change to the earliest possible change of the display content (latency). For each test and each device, participants performed 60 saccades between two targets 20° of visual angle apart. The targets were continuously visible in the HMD, and the saccades were instructed by an auditory cue. Data collection and eye tracking calibration were done using the recommended scripts for each device in Unity3D. The Vive Pro Eye was recorded twice, once using the SteamVR SDK and once using the Tobii XR SDK. Our results show clear differences between the HMDs. Delays ranged from 15 ms to 52 ms, and the latencies ranged from 45 ms to 81 ms. The Fove-0 appears to be the fastest device and best suited for gaze-contingent rendering.

Evaluation of 3D Pointing Accuracy in the Fovea and Periphery in Immersive Head-Mounted Display Environments.

The coupling between perception and action has seldom been explored in sophisticated motor behaviour such as 3D pointing. In this study, we investigated how 3D pointing accuracy, measured by a depth estimation task, could be affected by the target appearing in different visual eccentricities. Specifically, we manipulated the visual eccentricity of the target and its depth in virtual reality. Participants wore a head-mounted-display with an integrated eye-tracker and docked a cursor into a target. We adopted a within-participants factorial design with three variables. The first variable is Eccentricity: the location of the target on one of five horizontal eccentricities (left far periphery, left near periphery, foveal, right near periphery and right far periphery). The second variable is Depth at three levels and the third variable is Feedback Loop with two levels: open/closed. Eccentricity is refactored into Motion Correspondence between the starting location of the cursor and the target location with four levels: periphery to fovea, fovea to periphery, periphery to periphery, fovea to fovea. The results showed that the pointing accuracy is modulated mainly by the target locations rather than the initial locations of the effector (hand). Visible feedback during pointing improved performance.

Visual-Haptic Size Estimation in Peripersonal Space.

In perceptual psychology, estimations of visual depth and size under different spatial layouts have been extensively studied. However, research evidence in virtual environments (VE) is relatively lacking. The emergence of human-computer interaction (HCI) and virtual reality (VR) has raised the question of how human operators perform actions based on the estimation of visual properties in VR, especially when the sensory cues associated with the same object are conflicting. We report on an experiment in which participants compared the size of a visual sphere to a haptic sphere, belonging to the same object in a VE. The sizes from the visual and haptic modalities were either identical or conflicting (with visual size being larger than haptic size, or vice versa). We used three standard haptic references (small, medium, and large sizes) and asked participants to compare the visual sizes with the given reference, by method of constant stimuli. Results show a dominant functional priority of the visual size perception. Moreover, observers demonstrated a central tendency effect: over-estimation for smaller haptic sizes but under-estimation for larger haptic sizes. The results are in-line with previous studies in real environments (RE). We discuss the current findings in the framework of adaptation level theory for haptic size reference. This work provides important implications for the optimal design of human-computer interactions when integrating 3D visual-haptic information in a VE.

Stereoscopic Rendering via Goggles Elicits Higher Functional Connectivity During Virtual Reality Gaming.

Virtual reality (VR) simulates real-world scenarios by creating a sense of presence in its users. Such immersive scenarios lead to behavior that is more similar to that displayed in real world settings, which may facilitate the transfer of knowledge and skills acquired in VR to similar real world situations. VR has already been used in education, psychotherapy, rehabilitation and it comes as an appealing choice for training intervention purposes. The aim of the present study was to investigate to what extent VR technology for games presented via goggles can be used in a magnetic resonance imaging scanner (MRI), addressing the question of whether brain connectivity differs between VR stimulation via goggles and a presentation from a screen via mirror projection. Moreover, we wanted to investigate whether stereoscopic goggle stimulation, where both eyes receive different visual input, would elicit stronger brain connectivity than a stimulation in which both eyes receive the same visual input (monoscopic). To our knowledge, there is no previous research using games and functional connectivity (FC) in MRI to address this question. Multiple analyses approaches were taken so that different aspects of brain connectivity could be covered: fractional low-frequency fluctuation, independent component analysis (ICA), seed-based FC (SeedFC) and graph analysis. In goggle presentation (mono and stereoscopic) as contrasted to screen, we found differences in brain activation in left cerebellum and postcentral gyrus as well as differences in connectivity in the visual cortex and frontal inferior cortex [when focusing on the visual and default mode network (DMN)]. When considering connectivity in specific areas of interest, we found higher connectivity between bilateral superior frontal cortex and the temporal lobe, as well as bilateral inferior parietal cortex with right calcarine and right lingual cortex. Furthermore, we found superior frontal cortex and insula/putamen to be more strongly connected in goggle stereoscopic vs. goggle monoscopic, in line with our hypothesis. We assume that the condition that elicits higher brain connectivity values should be most suited for long-term brain training interventions given that, extended training under these conditions could permanently improve brain connectivity on a functional as well as on a structural level.

Gait Training in Virtual Reality: Short-Term Effects of Different Virtual Manipulation Techniques in Parkinson's Disease.

It is well documented that there is a strong relationship between gait asymmetry and the freezing of gait (FOG) in Parkinson's Disease. The purpose of this pilot study was to find a "virtual reality (VR)- based" gait manipulation strategy to improve gait symmetry by equalizing step length. Fifteen male PD patients (mean age of 67.6 years) with FOG were assessed on a GAITRite® walkway. Natural gait was compared with walking conditions during "VR-based" gait modulation tasks that aimed at equalizing gait symmetry using visual or proprioceptive signals. Compared to natural gait, VR manipulation tasks significantly increased step width and swing time variability for both body sides. Within the VR conditions, only the task with "proprioceptive-visual dissociation" by artificial backward shifting of the foot improved spatial asymmetry significantly with comparable step lengths of both sides. Specific, hypothesis-driven VR tasks represent an efficient tool to manipulate gait features as gait symmetry in PD potentially preventing FOG. This pilot study offers promising "VR-based" approaches for rehabilitative training strategies to achieve gait symmetry and prevent FOG.

Shrinking Circles: Adaptation to Increased Curvature Gain in Redirected Walking.

Real walking is the most natural way to locomote in virtual reality (VR), but a confined physical walking space limits its applicability. Redirected walking (RDW) is a collection of techniques to solve this problem. One of these techniques aims to imperceptibly rotate the user's view of the virtual scene in order to steer her along a confined path whilst giving the impression of walking in a straight line in a large virtual space. Measurement of perceptual thresholds for the detection of such a modified curvature gain have indicated a radius that is still larger than most room sizes. Since the brain is an adaptive system and thresholds usually depend on previous stimulations, we tested if prolonged exposure to an immersive virtual environment (IVE) with increased curvature gain produces adaptation to that gain and modifies thresholds such that, over time, larger curvature gains can be applied for RDW. Therefore, participants first completed a measurement of their perceptual threshold for curvature gain. In a second session, the same participants were exposed to an IVE with a constant curvature gain in which they walked between two targets for about 20 minutes. Afterwards, their perceptual thresholds were measured again. The results show that the psychometric curves shifted after the exposure session and perceptual thresholds for increased curvature gain further increased. The increase of the detection threshold suggests that participants adapt to the manipulation and stronger curvature gains can be applied in RDW, and therefore improves its applicability in such situations.

Analyses of Gait Parameters of Younger and Older Adults During (Non-)Isometric Virtual Walking.

Understanding real walking in virtual environments (VEs) is important for immersive experiences, allowing users to move through VEs in the most natural way. Previous studies have shown that basic implementations of real walking in virtual spaces, in which head-tracked movements are mapped isometrically to a VE, are not estimated as entirely natural. Instead, users estimate a virtual walking velocity as more natural when it is slightly increased compared to the user's physical locomotion. However, these findings have been reported in most cases only for young persons, e.g., students, whereas older adults are clearly underrepresented in such studies. Recently, virtual reality (VR) has received significant public and media attention. Therefore, it appears reasonable to assume that people at different ages will have access to VR, and might use this technology more and more in application scenarios such as rehabilitation or training. To better understand how people at different ages walk and perceive locomotion in VR, we have performed a study to investigate the effects of (non-)isometric mappings between physical movements and virtual motions in the VE on the walking biomechanics across generations, i.e., younger and older adults. Three primary domains (pace, base of support and phase) of spatio-temporal parameters were identified to evaluate gait performance. The results show that the older adults walked very similar in the real and VE in the pace and phasic domains, which differs from results found in younger adults. In contrast, the results indicate differences in terms of base of support domain parameters for both groups while walking within a VE and the real world. For non-isometric mappings, we found in both younger and older adults an increased divergence of gait parameters in all domains correlating with the up- or down-scaled velocity of visual self-motion feedback. The results provide important insights into the design of future VR applications for older adults in domains ranging from medicine and psychology to rehabilitation.

15 Years of Research on Redirected Walking in Immersive Virtual Environments.

Virtual reality users wearing head-mounted displays can experience the illusion of walking in any direction for infinite distance while, in reality, they are walking a curvilinear path in physical space. This is accomplished by introducing unnoticeable rotations to the virtual environment-a technique called redirected walking. This paper gives an overview of the research that has been performed since redirected walking was first practically demonstrated 15 years ago.

Detection Thresholds for Rotation and Translation Gains in 360° Video-Based Telepresence Systems.

Telepresence systems have the potential to overcome limits and distance constraints of the real-world by enabling people to remotely visit and interact with each other. However, current telepresence systems usually lack natural ways of supporting interaction and exploration of remote environments (REs). In particular, single webcams for capturing the RE provide only a limited illusion of spatial presence, and movement control of mobile platforms in today's telepresence systems are often restricted to simple interaction devices. One of the main challenges of telepresence systems is to allow users to explore a RE in an immersive, intuitive and natural way, e.g., by real walking in the user's local environment (LE), and thus controlling motions of the robot platform in the RE. However, the LE in which the user's motions are tracked usually provides a much smaller interaction space than the RE. In this context, redirected walking (RDW) is a very suitable approach to solve this problem. However, so far there is no previous work, which explored if and how RDW can be used in video-based 360° telepresence systems. In this article, we conducted two psychophysical experiments in which we have quantified how much humans can be unknowingly redirected on virtual paths in the RE, which are different from the physical paths that they actually walk in the LE. Experiment 1 introduces a discrimination task between local and remote translations, and in Experiment 2 we analyzed the discrimination between local and remote rotations. In Experiment 1 participants performed straightforward translations in the LE that were mapped to straightforward translations in the RE shown as 360° videos, which were manipulated by different gains. Then, participants had to estimate if the remotely perceived translation was faster or slower than the actual physically performed translation. Similarly, in Experiment 2 participants performed rotations in the LE that were mapped to the virtual rotations in a 360° video-based RE to which we applied different gains. Again, participants had to estimate whether the remotely perceived rotation was smaller or larger than the actual physically performed rotation. Our results show that participants are not able to reliably discriminate the difference between physical motion in the LE and the virtual motion from the 360° video RE when virtual translations are down-scaled by 5.8% and up-scaled by 9.7%, and virtual rotations are about 12.3% less or 9.2% more than the corresponding physical rotations in the LE.

Bending the Curve: Sensitivity to Bending of Curved Paths and Application in Room-Scale VR.

Redirected walking (RDW) promises to allow near-natural walking in an infinitely large virtual environment (VE) by subtle manipulations of the virtual camera. Previous experiments analyzed the human sensitivity to RDW manipulations by focusing on the worst-case scenario, in which users walk perfectly straight ahead in the VE, whereas they are redirected on a circular path in the real world. The results showed that a physical radius of at least 22 meters is required for undetectable RDW. However, users do not always walk exactly straight in a VE. So far, it has not been investigated how much a physical path can be bent in situations in which users walk a virtual curved path instead of a straight one. Such curved walking paths can be often observed, for example, when users walk on virtual trails, through bent corridors, or when circling around obstacles. In such situations the question is not, whether or not the physical path can be bent, but how much the bending of the physical path may vary from the bending of the virtual path. In this article, we analyze this question and present redirection by means of bending gains that describe the discrepancy between the bending of curved paths in the real and virtual environment. Furthermore, we report the psychophysical experiments in which we analyzed the human sensitivity to these gains. The results reveal encouragingly wider detection thresholds than for straightforward walking. Based on our findings, we discuss the potential of curved walking and present a first approach to leverage bent paths in a way that can provide undetectable RDW manipulations even in room-scale VR.

Who turned the clock? Effects of Manipulated Zeitgebers, Cognitive Load and Immersion on Time Estimation.

Current virtual reality (VR) technologies have enormous potential to allow humans to experience computer-generated immersive virtual environments (IVEs). Many of these IVEs support near-natural audiovisual stimuli similar to the stimuli generated in our physical world. However, decades of VR research have been devoted to exploring and understand differences between perception and action in such IVEs compared to real-world perception and action. Although, significant differences have been revealed for spatiotemporal perception between IVEs and the physical world such as distance underestimation, there is still a scarcity of knowledge about the reasons for such perceptual discrepancies, in particular regarding the perception of temporal durations in IVEs. In this article, we explore the effects of manipulated zeitgebers, cognitive load and immersion on time estimation as yet unexplored factors of spatiotemporal perception in IVEs. We present an experiment in which we analyze human sensitivity to temporal durations while experiencing an immersive head-mounted display (HMO) environment. We found that manipulations of external zeitgebers caused by a natural or unnatural movement of the virtual sun had a significant effect on time judgments. Moreover, using the dual-task paradigm the results show that increased spatial and verbal cognitive load resulted in a significant shortening of judged time as well as an interaction with the external zeitgebers. Finally, we discuss the implications for the design of near-natural computer-generated virtual worlds.