Try This for Size: Multi-Scale Teleportation in Immersive Virtual Reality.

The ability of a user to adjust their own scale while traveling through virtual environments enables them to inspect tiny features being ant-sized and to gain an overview of the surroundings as a giant. While prior work has almost exclusively focused on steering-based interfaces for multi-scale travel, we present three novel teleportation-based techniques that avoid continuous motion flow to reduce the risk of cybersickness. Our approaches build on the extension of known teleportation workflows and suggest specifying scale adjustments either simultaneously with, as a connected second step after, or separately from the user's new horizontal position. The results of a two-part user study with 30 participants indicate that the simultaneous and connected specification paradigms are both suitable candidates for effective and comfortable multi-scale teleportation with nuanced individual benefits. Scale specification as a separate mode, on the other hand, was considered less beneficial. We compare our findings to prior research and publish the executable of our user study to facilitate replication and further analyses.

IntenSelect+: Enhancing Score-Based Selection in Virtual Reality.

Object selection in virtual environments is one of the most common and recurring interaction tasks. Therefore, the used technique can critically influence a system's overall efficiency and usability. IntenSelect is a scoring-based selection-by-volume technique that was shown to offer improved selection performance over conventional raycasting in virtual reality. This initial method, however, is most pronounced for small spherical objects that converge to a point-like appearance only, is challenging to parameterize, and has inherent limitations in terms of flexibility. We present an enhanced version of IntenSelect called IntenSelect+ designed to overcome multiple shortcomings of the original IntenSelect approach. In an empirical within-subjects user study with 42 participants, we compared IntenSelect+ to IntenSelect and conventional raycasting on various complex object configurations motivated by prior work. In addition to replicating the previously shown benefits of IntenSelect over raycasting, our results demonstrate significant advantages of IntenSelect+ over IntenSelect regarding selection performance, task load, and user experience. We, therefore, conclude that IntenSelect+ is a promising enhancement of the original approach that enables faster, more precise, and more comfortable object selection in immersive virtual environments.

A lecturer's voice quality and its effect on memory, listening effort, and perception in a VR environment.

Many lecturers develop voice problems, such as hoarseness. Nevertheless, research on how voice quality influences listeners' perception, comprehension, and retention of spoken language is limited to a small number of audio-only experiments. We aimed to address this gap by using audio-visual virtual reality (VR) to investigate the impact of a lecturer's hoarseness on university students' heard text recall, listening effort, and listening impression. Fifty participants were immersed in a virtual seminar room, where they engaged in a Dual-Task Paradigm. They listened to narratives presented by a virtual female professor, who spoke in either a typical or hoarse voice. Simultaneously, participants performed a secondary task. Results revealed significantly prolonged secondary-task response times with the hoarse voice compared to the typical voice, indicating increased listening effort. Subjectively, participants rated the hoarse voice as more annoying, effortful to listen to, and impeding for their cognitive performance. No effect of voice quality was found on heard text recall, suggesting that, while hoarseness may compromise certain aspects of spoken language processing, this might not necessarily result in reduced information retention. In summary, our findings underscore the importance of promoting vocal health among lecturers, which may contribute to enhanced listening conditions in learning spaces.

Gaining the High Ground: Teleportation to Mid-Air Targets in Immersive Virtual Environments.

Most prior teleportation techniques in virtual reality are bound to target positions in the vicinity of selectable scene objects. In this paper, we present three adaptations of the classic teleportation metaphor that enable the user to travel to mid-air targets as well. Inspired by related work on the combination of teleports with virtual rotations, our three techniques differ in the extent to which elevation changes are integrated into the conventional target selection process. Elevation can be specified either simultaneously, as a connected second step, or separately from horizontal movements. A user study with 30 participants indicated a trade-off between the simultaneous method leading to the highest accuracy and the two-step method inducing the lowest task load as well as receiving the highest usability ratings. The separate method was least suitable on its own but could serve as a complement to one of the other approaches. Based on these findings and previous research, we define initial design guidelines for mid-air navigation techniques.

Advantages of a Training Course for Surgical Planning in Virtual Reality for Oral and Maxillofacial Surgery: Crossover Study.

BACKGROUND: As an integral part of computer-assisted surgery, virtual surgical planning (VSP) leads to significantly better surgery results, such as for oral and maxillofacial reconstruction with microvascular grafts of the fibula or iliac crest. It is performed on a 2D computer desktop screen (DS) based on preoperative medical imaging. However, in this environment, VSP is associated with shortcomings, such as a time-consuming planning process and the requirement of a learning process. Therefore, a virtual reality (VR)-based VSP application has great potential to reduce or even overcome these shortcomings due to the benefits of visuospatial vision, bimanual interaction, and full immersion. However, the efficacy of such a VR environment has not yet been investigated. OBJECTIVE: This study aimed to demonstrate the possible advantages of a VR environment through a substep of VSP, specifically the segmentation of the fibula (calf bone) and os coxae (hip bone), by conducting a training course in both DS and VR environments and comparing the results. METHODS: During the training course, 6 novices were taught how to use a software application in a DS environment (3D Slicer) and in a VR environment (Elucis) for the segmentation of the fibula and os coxae, and they were asked to carry out the maneuvers as accurately and quickly as possible. Overall, 13 fibula and 13 os coxae were segmented for each participant in both methods (VR and DS), resulting in 156 different models (78 fibula and 78 os coxae) per method (VR and DS) and 312 models in total. The individual learning processes in both environments were compared using objective criteria (time and segmentation performance) and self-reported questionnaires. The models resulting from the segmentation were compared mathematically (Hausdorff distance and Dice coefficient) and evaluated by 2 experienced radiologists in a blinded manner. RESULTS: A much faster learning curve was observed for the VR environment than the DS environment (ß=.86 vs ß=.25). This nearly doubled the segmentation speed (cm3/min) by the end of training, leading to a shorter time (P<.001) to reach a qualitative result. However, there was no qualitative difference between the models for VR and DS (P=.99). The VR environment was perceived by participants as more intuitive and less exhausting, and was favored over the DS environment. CONCLUSIONS: The more rapid learning process and the ability to work faster in the VR environment could save time and reduce the VSP workload, providing certain advantages over the DS environment.

Effect of head-mounted displays on students' acquisition of surgical suturing techniques compared to an e-learning and tutor-led course: a randomized controlled trial.

BACKGROUND: Although surgical suturing is one of the most important basic skills, many medical school graduates do not acquire sufficient knowledge of it due to its lack of integration into the curriculum or a shortage of tutors. E-learning approaches attempt to address this issue but still rely on the involvement of tutors. Furthermore, the learning experience and visual-spatial ability appear to play a critical role in surgical skill acquisition. Virtual reality head-mounted displays (HMDs) could address this, but the benefits of immersive and stereoscopic learning of surgical suturing techniques are still unclear. MATERIAL AND METHODS: In this multi-arm randomized controlled trial, 150 novices participated. Three teaching modalities were compared: an e-learning course (monoscopic), an HMD-based course (stereoscopic, immersive), both self-directed and a tutor-led course with feedback. Suturing performance was recorded by video camera both before and after course participation (>26 h of video material) and assessed in a blinded fashion using the Objective Structured Assessment of Technical Skills (OSATS) Global Rating Score (GRS). Furthermore, the optical flow of the videos was determined using an algorithm. The number of sutures performed was counted, the visual-spatial ability was measured with the Mental Rotation Test (MRT), and courses were assessed with questionnaires. RESULTS: Students' self-assessment in the HMD-based course was comparable to that of the tutor-led course and significantly better than in the e-learning course ( P =0.003). Course suitability was rated best for the tutor-led course ( x̄ =4.8), followed by the HMD-based ( x̄ =3.6) and e-learning ( x̄ =2.5) courses. The median ΔGRS between courses was comparable ( P =0.15) at 12.4 (95% CI 10.0-12.7) for the e-learning course, 14.1 (95% CI 13.0-15.0) for the HMD-based course, and 12.7 (95% CI 10.3-14.2) for the tutor-led course. However, the ΔGRS was significantly correlated with the number of sutures performed during the training session ( P =0.002), but not with visual-spatial ability ( P =0.615). Optical flow ( R2 =0.15, P <0.001) and the number of sutures performed ( R2 =0.73, P <0.001) can be used as additional measures to GRS. CONCLUSION: The use of HMDs with stereoscopic and immersive video provides advantages in the learning experience and should be preferred over a traditional web application for e-learning. Contrary to expectations, feedback is not necessary for novices to achieve a sufficient level in suturing; only the number of surgical sutures performed during training is a good determinant of competence improvement. Nevertheless, feedback still enhances the learning experience. Therefore, automated assessment as an alternative feedback approach could further improve self-directed learning modalities. As a next step, the data from this study could be used to develop such automated AI-based assessments.

Augmented Reality-Based Surgery on the Human Cadaver Using a New Generation of Optical Head-Mounted Displays: Development and Feasibility Study.

BACKGROUND: Although nearly one-third of the world's disease burden requires surgical care, only a small proportion of digital health applications are directly used in the surgical field. In the coming decades, the application of augmented reality (AR) with a new generation of optical-see-through head-mounted displays (OST-HMDs) like the HoloLens (Microsoft Corp) has the potential to bring digital health into the surgical field. However, for the application to be performed on a living person, proof of performance must first be provided due to regulatory requirements. In this regard, cadaver studies could provide initial evidence. OBJECTIVE: The goal of the research was to develop an open-source system for AR-based surgery on human cadavers using freely available technologies. METHODS: We tested our system using an easy-to-understand scenario in which fractured zygomatic arches of the face had to be repositioned with visual and auditory feedback to the investigators using a HoloLens. Results were verified with postoperative imaging and assessed in a blinded fashion by 2 investigators. The developed system and scenario were qualitatively evaluated by consensus interview and individual questionnaires. RESULTS: The development and implementation of our system was feasible and could be realized in the course of a cadaver study. The AR system was found helpful by the investigators for spatial perception in addition to the combination of visual as well as auditory feedback. The surgical end point could be determined metrically as well as by assessment. CONCLUSIONS: The development and application of an AR-based surgical system using freely available technologies to perform OST-HMD-guided surgical procedures in cadavers is feasible. Cadaver studies are suitable for OST-HMD-guided interventions to measure a surgical end point and provide an initial data foundation for future clinical trials. The availability of free systems for researchers could be helpful for a possible translation process from digital health to AR-based surgery using OST-HMDs in the operating theater via cadaver studies.

Probabilistic fibre tract analysis of cytoarchitectonically defined human inferior parietal lobule areas reveals similarities to macaques.

The human inferior parietal lobule (IPL) is a multimodal brain region, subdivided in several cytoarchitectonic areas which are involved in neural networks related to spatial attention, language, and higher motor processing. Tracer studies in macaques revealed differential connectivity patterns of IPL areas as the respective structural basis. Evidence for comparable differential fibre tracts of human IPL is lacking. Here, anatomical connectivity of five cytoarchitectonic human IPL areas to 64 cortical targets was investigated using probabilistic tractography. Connection likelihood was assessed by evaluating the number of traces between seed and target against the distribution of traces from that seed to voxels in the same distance as the target. The main fibre tract pattern shifted gradually from rostral to caudal IPL: Rostral areas were predominantly connected to somatosensory and superior parietal areas while caudal areas more strongly connected with auditory, anterior temporal and higher visual cortices. All IPL areas were strongly connected with inferior frontal, insular and posterior temporal areas. These results showed striking similarities with connectivity patterns in macaques, providing further evidence for possible homologies between these two species. This shift in fibre tract pattern supports a differential functional involvement of rostral (higher motor functions) and caudal IPL (spatial attention), with probable overlapping language involvement. The differential functional involvement of IPL areas was further supported by hemispheric asymmetries of connection patterns which showed left-right differences especially with regard to connections to sensorimotor, inferior frontal and temporal areas.

Representation of virtual arm movements in precuneus.

Arm movements can easily be adapted to different biomechanical constraints. However, the cortical representation of the processing of visual input and its transformation into motor commands remains poorly understood. In a visuo-motor dissociation paradigm, subjects were presented with a 3-D computer-graphical representation of a human arm, presenting movements of the subjects' right arm either as right or left arm. In order to isolate possible effects of coordinate transformations, coordinate mirroring at the body midline was implemented independently. In each of the resulting four conditions, 10 normal, right-handed subjects performed three runs of circular movements, while being scanned with O(15)-Butanol-PET. Kinematic analysis included orientation and accuracy of a fitted ellipsoid trajectory. Imaging analysis was performed with SPM 99 with activations threshold at P < 0.0001 (not corrected). The shape of the trajectory was dependent on the laterality of the arm, irrespective of movement mirroring, and accompanied by a robust activation difference in the contralateral precuneus. Movement mirroring decreased movement accuracy, which was related to increased activation in the left insula. Those two movement conditions that cannot be observed in reality were related to an activation focus at the left middle temporal gyrus, but showed no influence on movement kinematics. These findings demonstrate the prominent role of the precuneus for mediating visuo-motor transformations and have implications for the use of mirror therapy and virtual reality techniques, especially avatars, such as Nintendo Wii in neurorehabilitation.

Practical methods for designing medical training simulators.

We reviewed several approaches in literature used in the design process of medical training simulators. We have collected a set of useful practical methods which should help to efficiently derive a well-founded design for a specific surgical intervention in a structured manner.

Dissecting in silico: towards a taxonomy for medical simulators.

In this paper, we investigated several approaches in literature that classify different aspects of medical simulators. We have merged these definitions to form a structured taxonomy. This new taxonomy should facilitate the design of new medical simulators and allow to analyze and classify existing simulators, algorithms, toolkits and hardware.

Visualization of probabilistic fiber tracts in virtual reality.

Understanding the connectivity structure of the human brain is a fundamental prerequisite for the treatment of psychiatric or neurological diseases. Probabilistic tractography has become an established method to account for the inherent uncertainties of the actual course of fiber bundles in magnetic resonance imaging data. This paper presents a visualization system that addresses the assessment of fiber probabilities in relation to anatomical landmarks. We employ real-time transparent rendering strategy to display fiber tracts within their structural context in a virtual environment. Thereby, we not only emphasize spatial patterns but furthermore allow an interactive control over the amount of visible anatomical information.

Quantizing the void: extending Web3D for space-filling haptic meshes.

In this paper we summarize the progress of the Web3D scene graph model, and associated standards, specifically Extensible 3D (X3D) in the domain of medical simulation. Historically, the Web3D nodesets have focused on the representation and rendering of point, line or surface geometry. More recently, significant progress in X3D Volume rendering has been made available through the co-operative DICOM work item, n-Dimensional Presentation States. However, here we outline the need for a standard for simulation meshes and review several related approaches. As a result, we propose preliminary requirements for a simulation mesh standard and provide several use case scenarios of how Web3D and haptic technologies can aid the fulfillment of these requirements. We conclude with an X3D proposal to describe simulation meshes for soft (deformable) bodies.

Towards the visualization of spiking neurons in virtual reality.

This paper presents a prototype that addresses the visualization of the microscopic activity structure in the mammalian brain. Our approach displays the spiking behaviour of neurons in multiple layers based on large-scale simulations of the cortical microcircuit. We will apply this visualization to the activity of brain-scale simulations by coupling the microscopic structure with the macroscopic level. Thereby, we hope to convey an intuitive understanding of the concise interaction and the activity flow of pairs of distant brain areas.

Horizontal and vertical pseudoneglect in peri- and extrapersonal space.

The present study investigates the influence of depth on pseudoneglect in healthy young participants (n=18) within three-dimensional virtual space, by presenting a variation of the greyscales task and a landmark task, which were specifically matched for stimulus-response compatibility, as well as perceptual factors within and across the tasks. Tasks were presented in different depth locations (peripersonal, extrapersonal) and different orientations (horizontal, vertical) within three-dimensional virtual space, using virtual reality technique. A horizontal leftward bias (pseudoneglect) for both tasks was found, which was stronger in peripersonal than in extrapersonal space. For the vertical condition, an upward bias was observed in the greyscales task, but not in the landmark task. These results support the hypotheses of right hemispheric dominance for visual spatial attention and our study is the first to examine horizontal and vertical orienting biases with the greyscales task in peri- and extrapersonal space. Furthermore, the differences in attentional asymmetries with respect to depth suggest dissociable neural mechanisms for visual attentional processing in near and far space and the lack of significant correlations implies independence of horizontal and vertical stimulus processing.

Feature Tracking by Two-Step Optimization.

Tracking the temporal evolution of features in time-varying data is a key method in visualization. For typical feature definitions, such as vortices, objects are sparsely distributed over the data domain. In this paper, we present a novel approach for tracking both sparse and space-filling features. While the former comprise only a small fraction of the domain, the latter form a set of objects whose union covers the domain entirely while the individual objects are mutually disjunct. Our approach determines the assignment of features between two successive time-steps by solving two graph optimization problems. It first resolves one-to-one assignments of features by computing a maximum-weight, maximum-cardinality matching on a weighted bi-partite graph. Second, our algorithm detects events by creating a graph of potentially conflicting event explanations and finding a weighted, independent set in it. We demonstrate our method's effectiveness on synthetic and simulation data sets, the former of which enables quantitative evaluation because of the availability of ground-truth information. Here, our method performs on par or better than a well-established reference algorithm. In addition, manual visual inspection by our collaborators confirm the results' plausibility for simulation data.

Virtual reality-based regional anaesthesia simulator for axillary nerve blocks.

In this paper, we present a simulator for regional anaesthesia for nerve blocks in the axillary plexus region. We use a novel approach based on electric distance to simulate electronic impulse transmission through soft tissue. The traversal of electrons emitted from the needle tip is calculated by modified pathfinding algorithms. Kinematic algorithms visualize the motor response of the forearm by skeletal animation.

Low arousal modulates visuospatial attention in three-dimensional virtual space.

Clinical, experimental, and functional imaging studies suggest overlapping neuronal networks and functional interactions of alertness and visuospatial attention within the right hemisphere of the brain. To examine the interaction of arousal and visuospatial attention in peripersonal and extrapersonal virtual space, we tested 20 healthy male adults during 24 hr of sleep deprivation at four points during the night (9 p.m., 1 a.m., 5 a.m., and 9 a.m.). The main finding concerning covert orienting in a virtual environment is a highly significant slowing of reorientation toward the left visual hemifield in extrapersonal space due to decreased arousal. The results provide additional evidence for the proposed anatomical and functional overlap of the two attentional systems and indicate a modulation of visuospatial attention by the level of arousal in extrapersonal space.

Interactive blood damage analysis for ventricular assist devices.

Ventricular Assist Devices (VADs) support the heart in its vital task of maintaining circulation in the human body when the heart alone is not able to maintain a sufficient flow rate due to illness or degenerative diseases. However, the engineering of these devices is a highly demanding task. Advanced modeling methods and computer simulations allow the investigation of the fluid flow inside such a device and in particular of potential blood damage. In this paper we present a set of visualization methods which have been designed to specifically support the analysis of a tensor-based blood damage prediction model. This model is based on the tracing of particles through the VAD, for each of which the cumulative blood damage can be computed. The model's tensor output approximates a single blood cell's deformation in the flow field. The tensor and derived scalar data are subsequently visualized using techniques based on icons, particle visualization, and function plotting. All these techniques are accessible through a Virtual Reality-based user interface, which features not only stereoscopic rendering but also natural interaction with the complex three-dimensional data. To illustrate the effectiveness of these visualization methods, we present the results of an analysis session that was performed by domain experts for a specific data set for the MicroMed DeBakey VAD.

VIOLA-A Multi-Purpose and Web-Based Visualization Tool for Neuronal-Network Simulation Output.

Neuronal network models and corresponding computer simulations are invaluable tools to aid the interpretation of the relationship between neuron properties, connectivity, and measured activity in cortical tissue. Spatiotemporal patterns of activity propagating across the cortical surface as observed experimentally can for example be described by neuronal network models with layered geometry and distance-dependent connectivity. In order to cover the surface area captured by today's experimental techniques and to achieve sufficient self-consistency, such models contain millions of nerve cells. The interpretation of the resulting stream of multi-modal and multi-dimensional simulation data calls for integrating interactive visualization steps into existing simulation-analysis workflows. Here, we present a set of interactive visualization concepts called views for the visual analysis of activity data in topological network models, and a corresponding reference implementation VIOLA (VIsualization Of Layer Activity). The software is a lightweight, open-source, web-based, and platform-independent application combining and adapting modern interactive visualization paradigms, such as coordinated multiple views, for massively parallel neurophysiological data. For a use-case demonstration we consider spiking activity data of a two-population, layered point-neuron network model incorporating distance-dependent connectivity subject to a spatially confined excitation originating from an external population. With the multiple coordinated views, an explorative and qualitative assessment of the spatiotemporal features of neuronal activity can be performed upfront of a detailed quantitative data analysis of specific aspects of the data. Interactive multi-view analysis therefore assists existing data analysis workflows. Furthermore, ongoing efforts including the European Human Brain Project aim at providing online user portals for integrated model development, simulation, analysis, and provenance tracking, wherein interactive visual analysis tools are one component. Browser-compatible, web-technology based solutions are therefore required. Within this scope, with VIOLA we provide a first prototype.