Evaluation of an Audio-haptic Sensory Substitution Device for Enhancing Spatial Awareness for the Visually Impaired.

SIGNIFICANCE: Visually impaired participants were surprisingly fast in learning a new sensory substitution device, which allows them to detect obstacles within a 3.5-m radius and to find the optimal path in between. Within a few hours of training, participants successfully performed complex navigation as well as with the white cane. PURPOSE: Globally, millions of people live with vision impairment, yet effective assistive devices to increase their independence remain scarce. A promising method is the use of sensory substitution devices, which are human-machine interfaces transforming visual into auditory or tactile information. The Sound of Vision (SoV) system continuously encodes visual elements of the environment into audio-haptic signals. Here, we evaluated the SoV system in complex navigation tasks, to compare performance with the SoV system with the white cane, quantify training effects, and collect user feedback. METHODS: Six visually impaired participants received eight hours of training with the SoV system, completed a usability questionnaire, and repeatedly performed assessments, for which they navigated through standardized scenes. In each assessment, participants had to avoid collisions with obstacles, using the SoV system, the white cane, or both assistive devices. RESULTS: The results show rapid and substantial learning with the SoV system, with less collisions and higher obstacle awareness. After four hours of training, visually impaired people were able to successfully avoid collisions in a difficult navigation task as well as when using the cane, although they still needed more time. Overall, participants rated the SoV system's usability favorably. CONCLUSIONS: Contrary to the cane, the SoV system enables users to detect the best free space between objects within a 3.5-m (up to 10-m) radius and, importantly, elevated and dynamic obstacles. All in all, we consider that visually impaired people can learn to adapt to the haptic-auditory representation and achieve expertise in usage through well-defined training within acceptable time.

Effects of stimulus order on auditory distance discrimination of virtual nearby sound sources.

Stimulus order has been reported to affect perceived loudness. This letter investigates how temporal order affects distance discrimination of receding and approaching pairs of sound sources rendered binaurally in the anechoic near-field. Individual discrimination thresholds for different virtual locations were measured through an adaptive procedure. The threshold values show a bias toward approaching stimuli for closer reference distances (≤50 cm) and toward receding stimuli for farther reference distances (100 cm), but only when absolute intensity cues are available. The results show how an illusion of loudness can translate into an illusion of perceived relative distance.

On distance dependence of pinna spectral patterns in head-related transfer functions.

The aim of this letter is to address a little understood question in sound source localization: Can the distance of a near sound source affect our own perception of its elevation? The issue is studied by means of an objective analysis of a database of distance-dependent head-related transfer functions (HRTFs) of a KEMAR (Knowles Electronic Manikin for Acoustic Research) mannequin with different pinnae on a dense spatial grid. Iso-directional HRTFs are compared through spectral error metrics; results indicate significant distance-dependent HRTF modifications due to the pinna occur when the source is close to the interaural axis.

Cost-effective 3D scanning and printing technologies for outer ear reconstruction: current status.

Current 3D scanning and printing technologies offer not only state-of-the-art developments in the field of medical imaging and bio-engineering, but also cost and time effective solutions for surgical reconstruction procedures. Besides tissue engineering, where living cells are used, bio-compatible polymers or synthetic resin can be applied. The combination of 3D handheld scanning devices or volumetric imaging, (open-source) image processing packages, and 3D printers form a complete workflow chain that is capable of effective rapid prototyping of outer ear replicas. This paper reviews current possibilities and latest use cases for 3D-scanning, data processing and printing of outer ear replicas with a focus on low-cost solutions for rehabilitation engineering.

Effect of task-related continuous auditory feedback during learning of tracking motion exercises.

BACKGROUND: This paper presents the results of a set of experiments in which we used continuous auditory feedback to augment motor training exercises. This feedback modality is mostly underexploited in current robotic rehabilitation systems, which usually implement only very basic auditory interfaces. Our hypothesis is that properly designed continuous auditory feedback could be used to represent temporal and spatial information that could in turn, improve performance and motor learning. METHODS: We implemented three different experiments on healthy subjects, who were asked to track a target on a screen by moving an input device (controller) with their hand. Different visual and auditory feedback modalities were envisaged. The first experiment investigated whether continuous task-related auditory feedback can help improve performance to a greater extent than error-related audio feedback, or visual feedback alone. In the second experiment we used sensory substitution to compare different types of auditory feedback with equivalent visual feedback, in order to find out whether mapping the same information on a different sensory channel (the visual channel) yielded comparable effects with those gained in the first experiment. The final experiment applied a continuously changing visuomotor transformation between the controller and the screen and mapped kinematic information, computed in either coordinate system (controller or video), to the audio channel, in order to investigate which information was more relevant to the user. RESULTS: Task-related audio feedback significantly improved performance with respect to visual feedback alone, whilst error-related feedback did not. Secondly, performance in audio tasks was significantly better with respect to the equivalent sensory-substituted visual tasks. Finally, with respect to visual feedback alone, video-task-related sound feedback decreased the tracking error during the learning of a novel visuomotor perturbation, whereas controller-task-related sound feedback did not. This result was particularly interesting, as the subjects relied more on auditory augmentation of the visualized target motion (which was altered with respect to arm motion by the visuomotor perturbation), rather than on sound feedback provided in the controller space, i.e., information directly related to the effective target motion of their arm. CONCLUSIONS: Our results indicate that auditory augmentation of visual feedback can be beneficial during the execution of upper limb movement exercises. In particular, we found that continuous task-related information provided through sound, in addition to visual feedback can improve not only performance but also the learning of a novel visuomotor perturbation. However, error-related information provided through sound did not improve performance and negatively affected learning in the presence of the visuomotor perturbation.

Designing sensory-substitution devices: Principles, pitfalls and potential1.

An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.

Effects of complementary auditory feedback in robot-assisted lower extremity motor adaptation.

This study investigates how complementary auditory feedback may affect short-term gait modifications induced by four training sessions with a robotic exoskeleton. Healthy subjects walked on a treadmill and were instructed to match a modified gait pattern derived from their natural one, while receiving assistance by the robot (kinetic guidance). The main question we wanted to answer is whether the most commonly used combination of feedback (i.e., haptic and visual) could be either enhanced by adding auditory feedback or successfully substituted with a combination of kinetic guidance and auditory feedback. Participants were randomly assigned to one of four groups, all of which received kinetic guidance. The control group received additional visual feedback, while the three experimental groups were each provided with a different modality of auditory feedback. The third experimental group also received the same visual feedback as the control group. Differences among the training modalities in gait kinematics, timing and symmetry were assessed in three post-training sessions.

Improving robotics for neurorehabilitation: enhancing engagement, performance, and learning with auditory feedback.

This paper reports on an ongoing research collaboration between the University of Padua and the University of California Irvine, on the use of continuous auditory-feedback in robot-assisted neurorehabilitation of post-stroke patients. This feedback modality is mostly underexploited in current robotic rehabilitation systems, that usually implement very basic auditory feedback interfaces. The results of this research show that generating a proper sound cue during robot assisted movement training can help patients in improving engagement, performance and learning in the exercise.