Assessing Depth Perception in VR and Video See-Through AR: A Comparison on Distance Judgment, Performance, and Preference.

Spatial User Interfaces along the Reality-Virtuality continuum heavily depend on accurate depth perception. However, current display technologies still exhibit shortcomings in the simulation of accurate depth cues, and these shortcomings also vary between Virtual or Augmented Reality (VR, AR: eXtended Reality (XR) for short). This article compares depth perception between VR and Video See-Through (VST) AR. We developed a digital twin of an existing office room where users had top erform five depth-dependent tasks in VR and VST AR. Thirty-two participants took part in a user study using a 1 × 4 within-subjects design. Our results reveal higher misjudgment rates in VST AR due to conflicting depth cues between virtual and physical content. Increased head movements observed in participants were interpreted as a compensatory response to these conflicting cues. Furthermore, a longer task completion time in the VST AR condition indicates a lower task performance in VST AR. Interestingly, while participants rated the VR condition as easier and contrary to the increased misjudgments and lower performance with the VST AR display, a majority still expressed a preference for the VST AR experience. We discuss and explain these findings with the high visual dominance and referential power of the physical content in the VST AR condition, leading to a higher spatial presence and plausibility.

Handwriting for Text Input and the Impact of XR Displays, Surface Alignments, and Sentence Complexities.

Text input is desirable across various eXtended Reality (XR) use cases and is particularly crucial for knowledge and office work. This article compares handwriting text input between Virtual Reality (VR) and Video See-Through Augmented Reality (VST AR), facilitated by physically aligned and mid-air surfaces when writing simple and complex sentences. In a $2\times 2\times 2$ experimental design, 72 participants performed two ten-minute handwriting sessions, each including ten simple and ten complex sentences representing text input in real-world scenarios. Our developed handwriting application supports different XR displays, surface alignments, and handwriting recognition based on digital ink. We evaluated usability, user experience, task load, text input performance, and handwriting style. Our results indicate high usability with a successful transfer of handwriting skills to the virtual domain. XR displays and surface alignments did not impact text input speed and error rate. However, sentence complexities did, with participants achieving higher input speeds and fewer errors for simple sentences (17.85 WPM, 0.51% MSD ER) than complex sentences (15.07 WPM, 1.74% MSD ER). Handwriting on physically aligned surfaces showed higher learnability and lower physical demand, making them more suitable for prolonged handwriting sessions. Handwriting on mid-air surfaces yielded higher novelty and stimulation ratings, which might diminish with more experience. Surface alignments and sentence complexities significantly affected handwriting style, leading to enlarged and more connected cursive writing in both mid-air and for simple sentences. The study also demonstrated the benefits of using XR controllers in a pen-like posture to mimic styluses and pressure-sensitive tips on physical surfaces for input detection. We additionally provide a phrase set of simple and complex sentences as a basis for future text input studies, which can be expanded and adapted.

Visual Indicators Representing Avatars' Authenticity in Social Virtual Reality and Their Impacts on Perceived Trustworthiness.

Photorealistic avatars show great potential in social VR and VR collaboration. However, identity and privacy issues are threatening avatars' authenticity in social VR. In addition to the necessary authentication and protection, effective solutions are needed to convey avatars' authenticity status to users and thereby enhance the overall trustworthiness. We designed several visual indicators (VIs) using static or dynamic visual effects on photorealistic avatars and evaluated their effectiveness in visualizing avatars' authenticity status. In this study we explored suitable attributes and designs for conveying the authenticity of photorealistic avatars and influencing their perceived trustworthiness. Furthermore, we investigated how different interactivity levels influence their effectiveness (the avatar was either presented in a static image, an animated video clip, or an immersive virtual environment). Our findings showed that using a full name can increase trust, while most other VIs could decrease users' trust. We also found that interactivity levels significantly impacted users' trust and the effectiveness of VIs. Based on our results, we developed design guidelines for visual indicators as effective tools to convey authenticity, as a first step towards the improvement of trustworthiness in social VR with identity management.

The Impact of Avatar and Environment Congruence on Plausibility, Embodiment, Presence, and the Proteus Effect in Virtual Reality.

Many studies show the significance of the Proteus effect for serious virtual reality applications. The present study extends the existing knowledge by considering the relationship (congruence) between the self-embodiment (avatar) and the virtual environment. We investigated the impact of avatar and environment types and their congruence on avatar plausibility, sense of embodiment, spatial presence, and the Proteus effect. In a 2×2 between-subjects design, participants embodied either an avatar in sports- or business wear in a semantic congruent or incongruent environment while performing lightweight exercises in virtual reality. The avatar-environment congruence significantly affected the avatar's plausibility but not the sense of embodiment or spatial presence. However, a significant Proteus effect emerged only for participants who reported a high feeling of (virtual) body ownership, indicating that a strong sense of having and owning a virtual body is key to facilitating the Proteus effect. We discuss the results assuming current theories of bottom-up and top-down determinants of the Proteus effect and thus contribute to understanding its underlying mechanisms and determinants.

Measuring Interpersonal Trust towards Virtual Humans with a Virtual Maze Paradigm.

Virtual humans, including virtual agents and avatars, play an increasingly important role as VR technology advances. For example, virtual humans are used as digital bodies of users in social VR or as interfaces for AI assistants in online financing. Interpersonal trust is an essential prerequisite in real-life interactions, as well as in the virtual world. However, to date, there are no established interpersonal trust measurement tools specifically for virtual humans in virtual reality. This study fills the gap, by contributing a novel validated behavioural tool to measure interpersonal trust towards a specific virtual social interaction partner in social VR. This validated paradigm is inspired by a previously proposed virtual maze task that measures trust towards virtual characters. In the current study, a variant of this paradigm was implemented. The task of the users (the trustors) is to navigate through a maze in virtual reality, where they can interact with a virtual human (the trustee). They can choose to 1) ask for advice and 2) follow the advice from the virtual human if they want to. These measures served as behavioural measures of trust. We conducted a validation study with 70 participants in a between-subject design. The two conditions did not differ in the content of the advice but in the appearance, tone of voice and engagement of the trustees (alleged as avatars controlled by other participants). Results indicate that the experimental manipulation was successful, as participants rated the virtual human as more trustworthy in the trustworthy condition than in the untrustworthy condition. Importantly, this manipulation affected the trust behaviour of our participants, who, in the trustworthy condition, asked for advice more often and followed advice more often, indicating that the paradigm is sensitive to assessing interpersonal trust towards virtual humans. Thus, our paradigm can be used to measure differences in interpersonal trust towards virtual humans and may serve as a valuable research tool to study trust in virtual reality.

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality.

This article compares two state-of-the-art text input techniques between non-stationary virtual reality (VR) and video see-through augmented reality (VST AR) use-cases as XR display condition. The developed contact-based mid-air virtual tap and wordgesture (swipe) keyboard provide established support functions for text correction, word suggestions, capitalization, and punctuation. A user evaluation with 64 participants revealed that XR displays and input techniques strongly affect text entry performance, while subjective measures are only influenced by the input techniques. We found significantly higher usability and user experience ratings for tap keyboards compared to swipe keyboards in both VR and VST AR. Task load was also lower for tap keyboards. In terms of performance, both input techniques were significantly faster in VR than in VST AR. Further, the tap keyboard was significantly faster than the swipe keyboard in VR. Participants showed a significant learning effect with only ten sentences typed per condition. Our results are consistent with previous work in VR and optical see-through (OST) AR, but additionally provide novel insights into usability and performance of the selected text input techniques for VST AR. The significant differences in subjective and objective measures emphasize the importance of specific evaluations for each possible combination of input techniques and XR displays to provide reusable, reliable, and high-quality text input solutions. With our work, we form a foundation for future research and XR workspaces. Our reference implementation is publicly available to encourage replicability and reuse in future XR workspaces.

Exploring Plausibility and Presence in Mixed Reality Experiences.

Mixed Reality (MR) applications along Milgram's Reality-Virtuality (RV) continuum motivated a number of recent theories on potential constructs and factors describing MR experiences. This paper investigates the impact of incongruencies that are processed on different information processing layers (i.e., sensation/perception and cognition layer) to provoke breaks in plausibility. It examines the effects on spatial and overall presence as prominent constructs of Virtual Reality (VR). We developed a simulated maintenance application to test virtual electrical devices. Participants performed test operations on these devices in a counterbalanced, randomized 2x2 between-subject design in either VR as congruent or Augmented Reality (AR) as incongruent on the sensation/perception layer. Cognitive incongruence was induced by the absence of traceable power outages, decoupling perceived cause and effect after activating potentially defective devices. Our results indicate that the effects of the power outages differ significantly in the perceived plausibility and spatial presence ratings between VR and AR. Both ratings decreased for the AR condition (incongruent sensation/perception) compared to VR (congruent sensation/perception) for the congruent cognitive case but increased for the incongruent cognitive case. The results are discussed and put into perspective in the scope of recent theories of MR experiences.

Body and Time: Virtual Embodiment and its Effect on Time Perception.

This article explores the effect of one's body representation on time perception. Time Perception is modulated by a variety of factors including, e.g., the current situation or activity, it can display significant disturbances caused by psychological disorders, and it is influenced by emotional and interoceptive states, i.e., "the sense of the physiological condition of the body". We investigated this relation between one's own body and the perception of time in a novel Virtual Reality (VR) experiment explicitly fostering user activity. Forty-Eight participants randomly experienced different degrees of embodiment: i) without an avatar (low), ii) with hands (medium), and iii) with a high-quality avatar (high). Participants had to repeatedly activate a virtual lamp and estimate the duration of time intervals as well as judge the passage of time. Our results show a significant effect of embodiment on time perception: time passes slower in the low embodiment condition compared to the medium and high conditions. In contrast to prior work, the study provides missing evidence that this effect is independent of the level of activity of participants: In our task, users were prompted to repeatedly perform body actions, thereby ruling-out a potential influence of the level of activity. Importantly, duration judgements in both the millisecond and minute ranges seemed unaffected by variations in embodiment. Taken together, these results lead to a better understanding of the relationship between the body and time.

CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning.

Goal: This paper presents an immersive Virtual Reality (VR) system to analyze and train Executive Functions (EFs) of soccer players. EFs are important cognitive functions for athletes. They are a relevant quality that distinguishes amateurs from professionals. Method: The system is based on immersive technology, hence, the user interacts naturally and experiences a training session in a virtual world. The proposed system has a modular design supporting the extension of various so-called game modes. Game modes combine selected game mechanics with specific simulation content to target particular training aspects. The system architecture decouples selection/parameterization and analysis of training sessions via a coaching app from an Unity3D-based VR simulation core. Monitoring of user performance and progress is recorded by a database that sends the necessary feedback to the coaching app for analysis. Results: The system is tested for VR-critical performance criteria to reveal the usefulness of a new interaction paradigm in the cognitive training and analysis of EFs. Subjective ratings for overall usability show that the design as VR application enhances the user experience compared to a traditional desktop app; whereas the new, unfamiliar interaction paradigm does not negatively impact the effort for using the application. Conclusion: The system can provide immersive training of EF in a fully virtual environment, eliminating potential distraction. It further provides an easy-to-use analyzes tool to compare user but also an automatic, adaptive training mode.

A Fully-Immersive Virtual Reality Setup to Study Gait Modulation.

Objective: Gait adaptation to environmental challenges is fundamental for independent and safe community ambulation. The possibility of precisely studying gait modulation using standardized protocols of gait analysis closely resembling everyday life scenarios is still an unmet need. Methods: We have developed a fully-immersive virtual reality (VR) environment where subjects have to adjust their walking pattern to avoid collision with a virtual agent (VA) crossing their gait trajectory. We collected kinematic data of 12 healthy young subjects walking in real world (RW) and in the VR environment, both with (VR/A+) and without (VR/A-) the VA perturbation. The VR environment closely resembled the RW scenario of the gait laboratory. To ensure standardization of the obstacle presentation the starting time speed and trajectory of the VA were defined using the kinematics of the participant as detected online during each walking trial. Results: We did not observe kinematic differences between walking in RW and VR/A-, suggesting that our VR environment per se might not induce significant changes in the locomotor pattern. When facing the VA all subjects consistently reduced stride length and velocity while increasing stride duration. Trunk inclination and mediolateral trajectory deviation also facilitated avoidance of the obstacle. Conclusions: This proof-of-concept study shows that our VR/A+ paradigm effectively induced a timely gait modulation in a standardized immersive and realistic scenario. This protocol could be a powerful research tool to study gait modulation and its derangements in relation to aging and clinical conditions.

Breaking Plausibility Without Breaking Presence - Evidence For The Multi-Layer Nature Of Plausibility.

A novel theoretical model recently introduced coherence and plausibility as the essential conditions of XR experiences, challenging contemporary presence-oriented concepts. This article reports on two experiments validating this model, which assumes coherence activation on three layers (cognition, perception, and sensation) as the potential sources leading to a condition of plausibility and from there to other XR qualia such as presence or body ownership. The experiments introduce and utilize breaks in plausibility (in analogy to breaks in presence): We induce incoherence on the perceptual and the cognitive layer simultaneously by a simulation of object behaviors that do not conform to the laws of physics, i.e., gravity. We show that this manipulation breaks plausibility and hence confirm that it results in the desired effects in the theorized condition space but that the breaks in plausibility did not affect presence. In addition, we show that a cognitive manipulation by a storyline framing is too weak to successfully counteract the strong bottom-up inconsistencies. Both results are in line with the predictions of the recently introduced three-layer model of coherence and plausibility, which incorporates well-known top-down and bottom-up rivalries and its theorized increased independence between plausibility and presence.

A Structured Approach to Test the Signal Quality of Electroencephalography Measurements During Use of Head-Mounted Displays for Virtual Reality Applications.

Joint applications of virtual reality (VR) systems and electroencephalography (EEG) offer numerous new possibilities ranging from behavioral science to therapy. VR systems allow for highly controlled experimental environments, while EEG offers a non-invasive window to brain activity with a millisecond-ranged temporal resolution. However, EEG measurements are highly susceptible to electromagnetic (EM) noise and the influence of EM noise of head-mounted-displays (HMDs) on EEG signal quality has not been conclusively investigated. In this paper, we propose a structured approach to test HMDs for EM noise potentially harmful to EEG measures. The approach verifies the impact of HMDs on the frequency- and time-domain of the EEG signal recorded in healthy subjects. The verification task includes a comparison of conditions with and without an HMD during (i) an eyes-open vs. eyes-closed task, and (ii) with respect to the sensory- evoked brain activity. The approach is developed and tested to derive potential effects of two commercial HMDs, the Oculus Rift and the HTC Vive Pro, on the quality of 64-channel EEG measurements. The results show that the HMDs consistently introduce artifacts, especially at the line hum of 50 Hz and the HMD refresh rate of 90 Hz, respectively, and their harmonics. The frequency range that is typically most important in non-invasive EEG research and applications (<50 Hz) however, remained largely unaffected. Hence, our findings demonstrate that high-quality EEG recordings, at least in the frequency range up to 50 Hz, can be obtained with the two tested HMDs. However, the number of commercially available HMDs is constantly rising. We strongly suggest to thoroughly test such devices upfront since each HMD will most likely have its own EM footprint and this article provides a structured approach to implement such tests with arbitrary devices.

Embodiment in Virtual Reality Intensifies Emotional Responses to Virtual Stimuli.

Modulating emotional responses to virtual stimuli is a fundamental goal of many immersive interactive applications. In this study, we leverage the illusion of illusory embodiment and show that owning a virtual body provides means to modulate emotional responses. In a single-factor repeated-measures experiment, we manipulated the degree of illusory embodiment and assessed the emotional responses to virtual stimuli. We presented emotional stimuli in the same environment as the virtual body. Participants experienced higher arousal, dominance, and more intense valence in the high embodiment condition compared to the low embodiment condition. The illusion of embodiment thus intensifies the emotional processing of the virtual environment. This result suggests that artificial bodies can increase the effectiveness of immersive applications psychotherapy, entertainment, computer-mediated social interactions, or health applications.

User Responses to a Humanoid Robot Observed in Real Life, Virtual Reality, 3D and 2D.

Humanoid robots (i.e., robots with a human-like body) are projected to be mass marketed in the future in several fields of application. Today, however, user evaluations of humanoid robots are often based on mediated depictions rather than actual observations or interactions with a robot, which holds true not least for scientific user studies. People can be confronted with robots in various modes of presentation, among them (1) 2D videos, (2) 3D, i.e., stereoscopic videos, (3) immersive Virtual Reality (VR), or (4) live on site. A systematic investigation into how such differential modes of presentation influence user perceptions of a robot is still lacking. Thus, the current study systematically compares the effects of different presentation modes with varying immersive potential on user evaluations of a humanoid service robot. Participants (N = 120) observed an interaction between a humanoid service robot and an actor either on 2D or 3D video, via a virtual reality headset (VR) or live. We found support for the expected effect of the presentation mode on perceived immediacy. Effects regarding the degree of human likeness that was attributed to the robot were mixed. The presentation mode had no influence on evaluations in terms of eeriness, likability, and purchase intentions. Implications for empirical research on humanoid robots and practice are discussed.

Immersive virtual reality during gait rehabilitation increases walking speed and motivation: a usability evaluation with healthy participants and patients with multiple sclerosis and stroke.

BACKGROUND: The rehabilitation of gait disorders in patients with multiple sclerosis (MS) and stroke is often based on conventional treadmill training. Virtual reality (VR)-based treadmill training can increase motivation and improve therapy outcomes. The present study evaluated an immersive virtual reality application (using a head-mounted display, HMD) for gait rehabilitation with patients to (1) demonstrate its feasibility and acceptance and to (2) compare its short-term effects to a semi-immersive presentation (using a monitor) and a conventional treadmill training without VR to assess the usability of both systems and estimate the effects on walking speed and motivation. METHODS: In a within-subjects study design, 36 healthy participants and 14 persons with MS or stroke participated in each of the three experimental conditions (VR via HMD, VR via monitor, treadmill training without VR). RESULTS: For both groups, the walking speed in the HMD condition was higher than in treadmill training without VR and in the monitor condition. Healthy participants reported a higher motivation after the HMD condition as compared with the other conditions. Importantly, no side effects in the sense of simulator sickness occurred and usability ratings were high. No increases in heart rate were observed following the VR conditions. Presence ratings were higher for the HMD condition compared with the monitor condition for both user groups. Most of the healthy study participants (89%) and patients (71%) preferred the HMD-based training among the three conditions and most patients could imagine using it more frequently. CONCLUSIONS: For the first time, the present study evaluated the usability of an immersive VR system for gait rehabilitation in a direct comparison with a semi-immersive system and a conventional training without VR with healthy participants and patients. The study demonstrated the feasibility of combining a treadmill training with immersive VR. Due to its high usability and low side effects, it might be particularly suited for patients to improve training motivation and training outcome e. g. the walking speed compared with treadmill training using no or only semi-immersive VR. Immersive VR systems still require specific technical setup procedures. This should be taken into account for specific clinical use-cases during a cost-benefit assessment.

[Use of Virtual Reality as a Component of Acute and Chronic Pain Treatment]. / Virtual Reality als Baustein in der Behandlung akuter und chronischer Schmerzen.

Future or reality? Treating acute and chronic pain is a part of the daily routine of clinical anesthesiologists. Commonly used analgesics have unwanted side effects or may even be insufficient as in chronic pain treatment. Virtual Reality (VR) could be a promising new approach which offers noninvasive therapy options for the treatment of pain. In case of the opioid misuse the adjunctive treatment is mandatory. Various phenomena occur in VR, such as immersion, presence, embodiment and Proteus effect, which can cause a change in body awareness and behavior. Experimental and clinical studies already yielded some promising results for analgesic effects for acute and chronic pain conditions using VR simulation. Potential analgesic mechanisms include distraction, cognitive behavioral change, and distance from reality, leading to neurophysiological changes at the cortical level. The quality of the virtual environment, personalized avatars, as well as the possibility of interaction and multisensory input can increase immersion, which leads to a state of presence, and thus effective VR. VR can be used as an immersive extension or alternative to mirror therapy, especially for pain disorders such as complex regional pain syndrome (CRPS) or phantom limb pain. VR can be supplemented by gamification, which increases intrinsic motivation, well-being and adherence to therapy. In summary, VR could be an effective and realistic therapy option for acute and chronic pain in clinical and home settings in the future.

Construction of the Virtual Embodiment Questionnaire (VEQ).

User embodiment is important for many virtual reality (VR) applications, for example, in the context of social interaction, therapy, training, or entertainment. However, there is no data-driven and validated instrument to empirically measure the perceptual aspects of embodiment, necessary to reliably evaluate this important phenomenon. To provide a method to assess components of virtual embodiment in a reliable and consistent fashion, we constructed a Virtual Embodiment Questionnaire (VEQ). We reviewed previous literature to identify applicable constructs and questionnaire items, and performed a confirmatory factor analysis (CFA) on the data from three experiments ( N=196). The analysis confirmed three factors: (1) ownership of a virtual body, (2) agency over a virtual body, and (3) the perceived change in the body schema. A fourth study ( N=22) was conducted to confirm the reliability and validity of the scale, by investigating the impacts of latency and latency jitter present in the simulation. We present the proposed scale and study results and discuss resulting implications.

Self-organizing knowledge management might improve the quality of person-centered dementia care: A qualitative study.

BACKGROUND: In institutional dementia care, person-centered care improves care processes and the quality of life of residents. However, communication gaps impede the implementation of person-centered care in favor of routinized care. OBJECTIVE: We evaluated whether self-organizing knowledge management reduces communication gaps and improves the quality of person-centered dementia care. METHOD: We implemented a self-organizing knowledge management system. Eight significant others of residents with severe dementia and six professional caregivers used a mobile application for six months. We conducted qualitative interviews and focus groups afterward. MAIN FINDINGS: Participants reported that the system increased the quality of person-centered care, reduced communication gaps, increased the task satisfaction of caregivers and the wellbeing of significant others. CONCLUSIONS: Based on our findings, we develop the following hypotheses: self-organizing knowledge management might provide a promising tool to improve the quality of person-centered care. It might reduce communication barriers that impede person-centered care. It might allow transferring content-maintaining tasks from caregivers to significant others. Such distribution of tasks, in turn, might be beneficial for both parties. Furthermore, shared knowledge about situational features might guide person-centered interventions.

Sick Moves! Motion Parameters as Indicators of Simulator Sickness.

We explore motion parameters, more specifically gait parameters, as an objective indicator to assess simulator sickness in Virtual Reality (VR). We discuss the potential relationships between simulator sickness, immersion, and presence. We used two different camera pose (position and orientation) estimation methods for the evaluation of motion tasks in a large-scale VR environment: a simple model and an optimized model that allows for a more accurate and natural mapping of human senses. Participants performed multiple motion tasks (walking, balancing, running) in three conditions: a physical reality baseline condition, a VR condition with the simple model, and a VR condition with the optimized model. We compared these conditions with regard to the resulting sickness and gait, as well as the perceived presence in the VR conditions. The subjective measures confirmed that the optimized pose estimation model reduces simulator sickness and increases the perceived presence. The results further show that both models affect the gait parameters and simulator sickness, which is why we further investigated a classification approach that deals with non-linear correlation dependencies between gait parameters and simulator sickness. We argue that our approach could be used to assess and predict simulator sickness based on human gait parameters and we provide implications for future research.

Not Alone Here?! Scalability and User Experience of Embodied Ambient Crowds in Distributed Social Virtual Reality.

This article investigates performance and user experience in Social Virtual Reality (SVR) targeting distributed, embodied, and immersive, face-to-face encounters. We demonstrate the close relationship between scalability, reproduction accuracy, and the resulting performance characteristics, as well as the impact of these characteristics on users co-located with larger groups of embodied virtual others. System scalability provides a variable number of co-located avatars and Al-controlled agents with a variety of different appearances, including realistic-looking virtual humans generated from photogrammetry scans. The article reports on how to meet the requirements of embodied SVR with today's technical off-the-shelf solutions and what to expect regarding features, performance, and potential limitations. Special care has been taken to achieve low latencies and sufficient frame rates necessary for reliable communication of embodied social signals. We propose a hybrid evaluation approach which coherently relates results from technical benchmarks to subjective ratings and which confirms required performance characteristics for the target scenario of larger distributed groups. A user-study reveals positive effects of an increasing number of co-located social companions on the quality of experience of virtual worlds, i.e., on presence, possibility of interaction, and co-presence. It also shows that variety in avatar/agent appearance might increase eeriness but might also stimulate an increased interest of participants about the environment.