Co-Immersion in Audio Augmented Virtuality: The Case Study of a Static and Approximated Late Reverberation Algorithm.

In immersive Audio Augmented Reality, a virtual sound source should be indistinguishable from the existing real ones. This property can be evaluated with the co-immersion criterion, which encompasses scenes constituted by arbitrary configurations of real and virtual objects. Thus, we introduce the term Audio Augmented Virtuality (AAV) to describe a fully virtual environment consisting of auditory content captured from the real world, augmented by synthetic sound generation. We propose an experimental design in AAV investigating how simplified late reverberation (LR) affects the co-immersion of a sound source. Participants listened to simultaneous virtual speakers dynamically rendered through spatial Room Impulse Responses, and were asked to detect the presence of an impostor, i.e., a speaker rendered with one of two simplified LR conditions. Detection rates were found to be close to chance level, especially for one condition, suggesting a limited influence on co-immersion of the simplified LR in the evaluated AAV scenes. This methodology can be straightforwardly extended and applied to different acoustics scenes, complexities, i.e., the number of simultaneous speakers, and rendering parameters in order to further investigate the requirements for immersive audio technologies in AAR and AAV applications.

Action planning and affective states within the auditory peripersonal space in normal hearing and cochlear-implanted listeners.

Fast reaction to approaching stimuli is vital for survival. When sounds enter the auditory peripersonal space (PPS), sounds perceived as being nearer elicit higher motor cortex activation. There is a close relationship between motor preparation and the perceptual components of sounds, particularly of highly arousing sounds. Here we compared the ability to recognize, evaluate, and react to affective stimuli entering the PPS between 20 normal-hearing (NH, 7 women) and 10 cochlear-implanted (CI, 3 women) subjects. The subjects were asked to quickly flex their arm in reaction to positive (P), negative (N), and neutral (Nu) affective sounds ending virtually at five distances from their body. Pre-motor reaction time (pm-RT) was detected via electromyography from the postural muscles to measure action anticipation at the sound-stopping distance; the sounds were also evaluated for their perceived level of valence and arousal. While both groups were able to localize sound distance, only the NH group modulated their pm-RT based on the perceived sound distance. Furthermore, when the sound carried no affective components, the pm-RT to the Nu sounds was shorter compared to the P and the N sounds for both groups. Only the NH group perceived the closer sounds as more arousing than the distant sounds, whereas both groups perceived sound valence similarly. Our findings underline the role of emotional states in action preparation and describe the perceptual components essential for prompt reaction to sounds approaching the peripersonal space.

Pinna-related transfer functions and lossless wave equation using finite-difference methods: Validation with measurements.

Nowadays, wave-based simulations of head-related transfer functions (HRTFs) lack strong justifications to replace HRTF measurements. The main cause is the complex interactions between uncertainties and biases in both simulated and measured HRTFs. This paper deals with the validation of pinna-related high-frequency information in the ipsilateral directions-of-arrival, computed by lossless wave-based simulations with finite-difference models. A simpler yet related problem is given by the pinna-related transfer function (PRTF), which encodes the acoustical effects of only the external ear. Results stress that PRTF measurements are generally highly repeatable but not necessarily easily reproducible, leading to critical issues in terms of reliability for any ground truth condition. On the other hand, PRTF simulations exhibit an increasing uncertainty with frequency and grid-dependent frequency changes, which are here quantified analyzing the benefits in the use of a unique asymptotic solution. In this validation study, the employed finite-difference model accurately and reliably predict the PRTF magnitude mostly within ±1 dB up to ≈8 kHz and a space- and frequency-averaged spectral distortion within about 2 dB up to ≈ 18 kHz.

Superhuman Hearing - Virtual Prototyping of Artificial Hearing: a Case Study on Interactions and Acoustic Beamforming.

Directivity and gain in microphone array systems for hearing aids or hearable devices allow users to acoustically enhance the information of a source of interest. This source is usually positioned directly in front. This feature is called acoustic beamforming. The current study aimed to improve users' interactions with beamforming via a virtual prototyping approach in immersive virtual environments (VEs). Eighteen participants took part in experimental sessions composed of a calibration procedure and a selective auditory attention voice-pairing task. Eight concurrent speakers were placed in an anechoic environment in two virtual reality (VR) scenarios. The scenarios were a purely virtual scenario and a realistic 360° audio-visual recording. Participants were asked to find an individual optimal parameterization for three different virtual beamformers: (i) head-guided, (ii) eye gaze-guided, and (iii) a novel interaction technique called dual beamformer, where head-guided is combined with an additional hand-guided beamformer. None of the participants were able to complete the task without a virtual beamformer (i.e., in normal hearing condition) due to the high complexity introduced by the experimental design. However, participants were able to correctly pair all speakers using all three proposed interaction metaphors. Providing superhuman hearing abilities in the form of a dual acoustic beamformer guided by head and hand movements resulted in statistically significant improvements in terms of pairing time, suggesting the task-relevance of interacting with multiple points of interests.

Cervical Spine Motion During Vehicle Extrication of Healthy Volunteers.

Objective: Prehospital spinal motion restriction as a prevention technique for secondary neurological injury is a key principle in emergency medicine. Our aim was to evaluate the effectiveness of different cervical spinal cord motion restriction techniques of awake and cooperative healthy volunteers during extrication.Methods: Twenty-three healthy volunteers were asked to exit a car (unassisted) with a rigid cervical collar (CC condition) or without it (autonomous exit: AE; instructed exit: IE); they were also extricated by two rescuers after setting a rigid cervical collar and by using an extrication device (CC + XT condition). Eight 3 D infrared cameras were calibrated around the vehicle to measure cervical spine angle, angular speed and acceleration in the sagittal plane. Surface wireless EMG electrodes were used to record superior trapezius, erector spinae and rectus abdominis muscle activity. All measures were recorded during two phases: device positioning (maneuver) and vehicle exiting.Results: The lowest range of motion was observed in CC during maneuver and exit (about 17°), the greatest in AE and IE (about 45°); when the extrication device was utilized along with the cervical collar (CC + XT) an increase, rather than a further decrease, in the range of motion was observed (about 25° during maneuver and exit). Larger values of angular speed and acceleration were observed in CC + XT when compared to CC, both during maneuver and exit (p < 0.001). The lowest EMG activity was observed during maneuver in CC and CC + XT; during exit a lower EMG activity was observed in CC + XT compared to CC (p < 0.001). Thus, when an extrication device is utilized (CC + XT), a lower active control of the cervical spine region is associated with faster and more brisk movements of the cervical spine compared to CC alone.Conclusions: Our findings support the idea that spinal motion restriction via rigid cervical collar of awake and cooperative trauma patients is effective in reducing cervical spine motion in the sagittal plane during vehicle extrication.

Pinna-related transfer functions and lossless wave equation using finite-difference methods: Verification and asymptotic solution.

A common approach when employing discrete mathematical models is to assess the reliability and credibility of the computation of interest through a process known as solution verification. Present-day computed head-related transfer functions (HRTFs) seem to lack robust and reliable assessments of the numerical errors embedded in the results which makes validation of wave-based models difficult. This process requires a good understanding of the involved sources of error which are systematically reviewed here. The current work aims to quantify the pinna-related high-frequency computational errors in the context of HRTFs and wave-based simulations with finite-difference models. As a prerequisite for solution verification, code verification assesses the reliability of the proposed implementation. In this paper, known and manufactured formal solutions are used and tailored for the wave equation and frequency-independent boundary conditions inside a rectangular room of uniform acoustic wall-impedance. Asymptotic estimates for pinna acoustics are predicted in the frequency domain based on regression models and a convergence study on sub-millimeter grids. Results show an increasing uncertainty with frequency and a significant frequency-dependent change among computations on different grids.

Auditory Feedback for Navigation with Echoes in Virtual Environments: Training Procedure and Orientation Strategies.

Being able to hear objects in an environment, for example using echolocation, is a challenging task. The main goal of the current work is to use virtual environments (VEs) to train novice users to navigate using echolocation. Previous studies have shown that musicians are able to differentiate sound pulses from reflections. This paper presents design patterns for VE simulators for both training and testing procedures, while classifying users' navigation strategies in the VE. Moreover, the paper presents features that increase users' performance in VEs. We report the findings of two user studies: a pilot test that helped improve the sonic interaction design, and a primary study exposing participants to a spatial orientation task during four conditions which were early reflections (RF), late reverberation (RV), early reflections-reverberation (RR) and visual stimuli (V). The latter study allowed us to identify navigation strategies among the users. Some users (10/26) reported an ability to create spatial cognitive maps during the test with auditory echoes, which may explain why this group performed better than the remaining participants in the RR condition.

Sonic Interactions in Virtual Reality: State of the Art, Current Challenges, and Future Directions.

A high-fidelity but efficient sound simulation is an essential element of any VR experience. Many of the techniques used in virtual acoustics are graphical rendering techniques suitably modified to account for sound generation and propagation. In recent years, several advances in hardware and software technologies have been facilitating the development of immersive interactive sound-rendering experiences. In this article, we present a review of the state of the art of such simulations, with a focus on the different elements that, combined, provide a complete interactive sonic experience. This includes physics-based simulation of sound effects and their propagation in space together with binaural rendering to simulate the position of sound sources. We present how these different elements of the sound design pipeline have been addressed in the literature, trying to find the trade-off between accuracy and plausibility. Recent applications and current challenges are also presented.

Influence of voxelization on finite difference time domain simulations of head-related transfer functions.

The scattering around the human pinna that is captured by the Head-Related Transfer Functions (HRTFs) is a complex problem that creates uncertainties in both acoustical measurements and simulations. Within the simulation framework of Finite Difference Time Domain (FDTD) with axis-aligned staircase boundaries resulting from a voxelization process, the voxelization-based uncertainty propagating in the HRTF-captured sound field is quantified for one solid and two surface voxelization algorithms. Simulated results utilizing a laser-scanned mesh of Knowles Electronics Manikin for Acoustic Research (KEMAR) show that in the context of complex geometries with local topology comparable to grid spacing such as the human pinna, the voxelization-related uncertainties in simulations emerge at lower frequencies than the generally used accuracy bandwidths. Numerical simulations show that the voxelization process induces both random error and algorithm-dependent bias in the simulated HRTF spectral features. Frequencies fr below which the random error is bounded by various dB thresholds are estimated and predicted. Particular shortcomings of the used voxelization algorithms are identified and the influence of the surface impedance on the induced errors is studied. Simulations are also validated against measurements.

Absence of modulatory action on haptic height perception with musical pitch.

Although acoustic frequency is not a spatial property of physical objects, in common language, pitch, i.e., the psychological correlated of frequency, is often labeled spatially (i.e., "high in pitch" or "low in pitch"). Pitch-height is known to modulate (and interact with) the response of participants when they are asked to judge spatial properties of non-auditory stimuli (e.g., visual) in a variety of behavioral tasks. In the current study we investigated whether the modulatory action of pitch-height extended to the haptic estimation of height of a virtual step. We implemented a HW/SW setup which is able to render virtual 3D objects (stair-steps) haptically through a PHANTOM device, and to provide real-time continuous auditory feedback depending on the user interaction with the object. The haptic exploration was associated with a sinusoidal tone whose pitch varied as a function of the interaction point's height within (i) a narrower and (ii) a wider pitch range, or (iii) a random pitch variation acting as a control audio condition. Explorations were also performed with no sound (haptic only). Participants were instructed to explore the virtual step freely, and to communicate height estimation by opening their thumb and index finger to mimic the step riser height, or verbally by reporting the height in centimeters of the step riser. We analyzed the role of musical expertise by dividing participants into non-musicians and musicians. Results showed no effects of musical pitch on high-realistic haptic feedback. Overall there is no difference between the two groups in the proposed multimodal conditions. Additionally, we observed a different haptic response distribution between musicians and non-musicians when estimations of the auditory conditions are matched with estimations in the no sound condition.