Enhancing learning outcomes through multisensory integration: A fMRI study of audio-visual training in virtual reality.

The integration of information from different sensory modalities is a fundamental process that enhances perception and performance in real and virtual environments (VR). Understanding these mechanisms, especially during learning tasks that exploit novel multisensory cue combinations provides opportunities for the development of new rehabilitative interventions. This study aimed to investigate how functional brain changes support behavioural performance improvements during an audio-visual (AV) learning task. Twenty healthy participants underwent a 30 min daily VR training for four weeks. The task was an AV adaptation of a 'scanning training' paradigm that is commonly used in hemianopia rehabilitation. Functional magnetic resonance imaging (fMRI) and performance data were collected at baseline, after two and four weeks of training, and four weeks post-training. We show that behavioural performance, operationalised as mean reaction time reduction in VR, significantly improves. In separate tests in a controlled laboratory environment, we showed that the behavioural performance gains in the VR training environment transferred to a significant mean RT reduction for the trained AV voluntary task on a computer screen. Enhancements were observed in both the visual-only and AV conditions, with the latter demonstrating a faster response time supported by the presence of audio cues. The behavioural learning effect also transfers to two additional tasks that were tested: a visual search task and an involuntary visual task. Our fMRI results reveal an increase in functional activation (BOLD signal) in multisensory brain regions involved in early-stage AV processing: the thalamus, the caudal inferior parietal lobe and cerebellum. These functional changes were only observed for the trained, multisensory, task and not for unimodal visual stimulation. Functional activation changes in the thalamus were significantly correlated to behavioural performance improvements. This study demonstrates that incorporating spatial auditory cues to voluntary visual training in VR leads to augmented brain activation changes in multisensory integration, resulting in measurable performance gains across tasks. The findings highlight the potential of VR-based multisensory training as an effective method for enhancing cognitive function and as a potentially valuable tool in rehabilitative programmes.

Audio-visual stimulation for visual compensatory functions in stroke survivors with visual field defect: a systematic review.

BACKGROUND: Hemianopia is a complete or partial blindness in the visual fields of both eyes, commonly caused by cerebral infarction. It has been hypothesized that systematic audio-visual (AV) stimulation of the blind hemifield can improve accuracy and search times, probably due to the stimulation of bimodal representations in the superior colliculus (SC), an important multisensory structure involved in both the initiation and execution of saccades. METHODS: A narrative synthesis of the findings is presented to highlight how AV rehabilitation impacts on patients with hemianopia including visual oculomotor function, functional ability in activities of daily living, hemianopic dyslexia, visual scanning and searching tasks, maintaining of functional ability post training and the effect on brain multisensory integration by using neuroimaging. RESULTS: Sixteen studies were included (fourteen articles (188 participants) and two literature reviews). Results were grouped into AV training of hemianopia in adults and in children and then further grouped according to the AV task type: tasks measuring the training effects by comparing visual stimulation training to audio-visual training, localization abilities in homonymous hemianopia (HH) and AV integration in patients with HH. CONCLUSION: Systematic AV training may improve the processing of visual information by recruiting subcortical pathways, and because most of the patients with visual cortex damage have an intact SC, it might be useful to use the bimodal AV training to activate retinotectal functions. Nevertheless, the underlying mechanisms supporting the reported positive effects are not currently understood. Systematic functional and/or structural imaging studies may help in understanding the underlying mechanism and inform the design of optimal training paradigms.

What German dentists choose for their teeth: A Web-based survey of molar restorations and their longevity.

STATEMENT OF PROBLEM: Which restorative material is best suited for treating the posterior molar region is unclear. As dentists presumably choose the most appropriate restoration, German dentists were asked how their own molars were treated. PURPOSE: The purpose of this survey study was to examine molar restorations and their durability in German dentists. MATERIAL AND METHODS: In the official journal of the Federal Dental Association, German dentists were asked to participate in an online survey that gathered demographic data and information on the type and durability of their molar restorations. To reduce selection bias, the data were weighted for region, sex, and age of the dentist. RESULTS: The data set consisted of 1719 molars from 288 dentists. Restorations included gold inlays (25%), composite resin (24.3%), amalgam (11.8%), ceramic inlays (5.4%), glass ionomer cement or compomer (0.8%), gold crowns (21.8%), ceramic crowns (6.6%), and metal-ceramic crowns (4.3%). Notable differences were identified based on the sex, age, and region of the dentist. Women selected increased esthetic options, as did young dentists. Restorations made of gold, amalgam, and base metal had the highest longevity, at more than 20 years CONCLUSIONS: Among German dentists, restorations with metallic materials dominated in molars, with gold accounting for the largest share. In recent years, the proportion of tooth-colored restorations has increased, and in particular, young dentists use composite resin and ceramics.

Perceived Audio-Visual Simultaneity Is Recalibrated by the Visual Intensity of the Preceding Trial.

A vital heuristic used when making judgements on whether audio-visual signals arise from the same event, is the temporal coincidence of the respective signals. Previous research has highlighted a process, whereby the perception of simultaneity rapidly recalibrates to account for differences in the physical temporal offsets of stimuli. The current paper investigated whether rapid recalibration also occurs in response to differences in central arrival latencies, driven by visual-intensity-dependent processing times. In a behavioural experiment, observers completed a temporal-order judgement (TOJ), simultaneity judgement (SJ) and simple reaction-time (RT) task and responded to audio-visual trials that were preceded by other audio-visual trials with either a bright or dim visual stimulus. It was found that the point of subjective simultaneity shifted, due to the visual intensity of the preceding stimulus, in the TOJ, but not SJ task, while the RT data revealed no effect of preceding intensity. Our data therefore provide some evidence that the perception of simultaneity rapidly recalibrates based on stimulus intensity.

Veterinary students' proximity to and interpretation of a simulated "aggressive" dog before and after training.

Dog "aggression" in the veterinary practice is commonplace. Therefore, student knowledge and education about dog behaviour and the ability to interpret "aggressive" behaviour is important from a human injury prevention and dog welfare perspective. The study aimed to compare first-year veterinary students' perceived safest proximity to both an "aggressive" and non-reactive simulated dog, both before and after a teaching intervention about canine behaviour and a handling practical. It also examined student confidence and their ability to identify "aggressive" behaviours. Forty first year veterinary students took part in two surveys. Each survey included two videos: one of a simulated dog displaying "aggressive" behaviour, based on the 'Canine Ladder of Aggression'; and another displaying non-reactive (passive behaviours without reaction to the participants) behaviours. Each video depicted the slow and consistent approach towards the virtual dog within a virtual indoor environment, and participants were asked to press stop if or when they would stop approaching the dog. In the "aggressive" scenario, there was a reduction in the approach-stop time from survey 1 (median = 17.8 s) to survey 2 (median = 15.2 s) in the intervention group (p = 0.018) but not in the control group (p = 0.147). Regarding confidence, there was a significant increase in the self-reported confidence rating relating to a participant's ability to interpret canine behaviour in both the control (p = 0.011) and intervention (p = 0.003). In conclusion, these results indicate that students using approach-stop videos stayed further away from an "aggressive" virtual dog model if they had undertaken a canine behaviour educational intervention. This novel approach has the potential for further use in teaching and assessment of student knowledge and behaviour which may otherwise be difficult to demonstrate.

Rapid neural changes during novel speech-sound learning: An fMRI and DTI study.

While the functional and microstructural changes that occur when we learn new language skills are well documented, relatively little is known about the time course of these changes. Here a combined functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) study that tracks neural change over three days of learning Arabic phonetic categorization as a new language (L-training) is presented. Twenty adult native English-speaking (L-native) participants are scanned before and after training to perceive and produce L-training phonetic contrasts for one hour on three consecutive days. A third (Chinese) language is used as a control language (L-control). Behavioral results show significant performance improvement for L-training in both learnt tasks; the perception and production task. Imaging analysis reveals that, training-related hemodynamic fMRI signal and fractional anisotropy (FA) value increasing can be observed, in the left inferior frontal gyrus (LIFG) and positively correlated with behavioral improvement. Moreover, post training functional connectivity findings show a significant increasing between LIFG and left inferior parietal lobule for L-training. These results indicate that three hours of phonetic categorization learning causes functional and microstructural changes that are typically associated with much more long-term learning.

Premotor cortex is sensitive to auditory-visual congruence for biological motion.

The auditory and visual perception systems have developed special processing strategies for ecologically valid motion stimuli, utilizing some of the statistical properties of the real world. A well-known example is the perception of biological motion, for example, the perception of a human walker. The aim of the current study was to identify the cortical network involved in the integration of auditory and visual biological motion signals. We first determined the cortical regions of auditory and visual coactivation (Experiment 1); a conjunction analysis based on unimodal brain activations identified four regions: middle temporal area, inferior parietal lobule, ventral premotor cortex, and cerebellum. The brain activations arising from bimodal motion stimuli (Experiment 2) were then analyzed within these regions of coactivation. Auditory footsteps were presented concurrently with either an intact visual point-light walker (biological motion) or a scrambled point-light walker; auditory and visual motion in depth (walking direction) could either be congruent or incongruent. Our main finding is that motion incongruency (across modalities) increases the activity in the ventral premotor cortex, but only if the visual point-light walker is intact. Our results extend our current knowledge by providing new evidence consistent with the idea that the premotor area assimilates information across the auditory and visual modalities by comparing the incoming sensory input with an internal representation.

A pilot study investigating human behaviour towards DAVE (Dog Assisted Virtual Environment) and interpretation of non-reactive and aggressive behaviours during a virtual reality exploration task.

Dog aggression is a public health concern because dog bites often lead to physical and psychological trauma in humans. It is also a welfare concern for dogs. To prevent aggressive behaviours, it is important to understand human behaviour towards dogs and our ability to interpret signs of dog aggression. This poses ethical challenges for humans and dogs. The aim of this study was to introduce, describe and pilot test a virtual reality dog model (DAVE (Dog Assisted Virtual Environment)). The Labrador model has two different modes displaying aggressive and non-reactive non-aggressive behaviours. The aggressive behaviours displayed are based on the current understanding of canine ethology and expert feedback. The objective of the study was to test the recognition of dog behaviour and associated human approach and avoidance behaviour. Sixteen university students were recruited via an online survey to participate in a practical study, and randomly allocated to two experimental conditions, an aggressive followed by a non-reactive virtual reality model (group AN) or vice versa (group NA). Participants were instructed to 'explore the area' in each condition, followed by a survey. A Wilcoxon and Mann Whitney U test was used to compare the closest distance to the dog within and between groups respectively. Participants moved overall significantly closer to the non-reactive dog compared to the aggressive dog (p≤0.001; r = 0.8). Descriptions of the aggressive dog given by participants often used motivational or emotional terms. There was little evidence of simulator sickness and presence scores were high indicating sufficient immersion in the virtual environment. Participants appeared to perceive the dog as realistic and behaved and interacted with the dog model in a manner that might be expected during an interaction with a live dog. This study also highlights the promising results for the potential future use of virtual reality in behavioural research (i.e., human-dog interactions), education (i.e. safety around dogs) and psychological treatment (e.g. dog phobia treatment).

Interactions between auditory and visual semantic stimulus classes: evidence for common processing networks for speech and body actions.

Incongruencies between auditory and visual signals negatively affect human performance and cause selective activation in neuroimaging studies; therefore, they are increasingly used to probe audiovisual integration mechanisms. An open question is whether the increased BOLD response reflects computational demands in integrating mismatching low-level signals or reflects simultaneous unimodal conceptual representations of the competing signals. To address this question, we explore the effect of semantic congruency within and across three signal categories (speech, body actions, and unfamiliar patterns) for signals with matched low-level statistics. In a localizer experiment, unimodal (auditory and visual) and bimodal stimuli were used to identify ROIs. All three semantic categories cause overlapping activation patterns. We find no evidence for areas that show greater BOLD response to bimodal stimuli than predicted by the sum of the two unimodal responses. Conjunction analysis of the unimodal responses in each category identifies a network including posterior temporal, inferior frontal, and premotor areas. Semantic congruency effects are measured in the main experiment. We find that incongruent combinations of two meaningful stimuli (speech and body actions) but not combinations of meaningful with meaningless stimuli lead to increased BOLD response in the posterior STS (pSTS) bilaterally, the left SMA, the inferior frontal gyrus, the inferior parietal lobule, and the anterior insula. These interactions are not seen in premotor areas. Our findings are consistent with the hypothesis that pSTS and frontal areas form a recognition network that combines sensory categorical representations (in pSTS) with action hypothesis generation in inferior frontal gyrus/premotor areas. We argue that the same neural networks process speech and body actions.

Electrophysiological correlates of facial configuration and audio-visual congruency: evidence that face processing is a visual rather than a multisensory task.

Behavioural, neuroimaging and lesion studies show that face processing has a special role in human perception. The purpose of this EEG study was to explore whether auditory information influences visual face perception. We employed a 2 × 2 factorial design and presented subjects with visual stimuli that could be cartoon faces or scrambled faces where size changes of one of the components, the mouth in the face condition, was either congruent or incongruent with the amplitude modulation of a simultaneously presented auditory signal. Our data show a significant main effect for signal congruence at an ERP peak around 135 ms and a significant main effect of face configuration at around 200 ms. The timing and scalp topology of both effects corresponds well to previously reported data on the integration of non-redundant audio-visual stimuli and face-selective processing. Our analysis did not show any significant statistical interactions. This double disassociation suggests that the early component, at 135 ms, is sensitive to auditory-visual congruency but not to facial configuration and that the later component is sensitive to facial configuration but not to AV congruency. We conclude that facial configurational processing is not influenced by the congruence of simultaneous auditory signals and is independent from featural processing where we see evidence for multisensory integration.

The classic: The architecture of the trabecular bone (tenth contribution on the mechanics of the human skeletal framework).

Hermann von Meyer may rightfully be considered the original proposer of the concept of trabecular architecture following patterns suggesting the directions of principle compressive and tensile stresses in a similarly shaped trabecular structure. Until the mid 19th Century, few had observed trabecular architecture, and when depicted was generally considered to have little regularity. In the 1830s Bourgery, Ward, and Wyman independently described the regularity of trabecular architecture, but according to Koch (1917) the proposed explanations were in error or overly simplified. Karl Culmann, an engineer and the developer of "graphic statics," attended a lecture of von Meyer and made the connection, which was then developed by the latter in this seminal paper. We present the paper in translation here. The original German article entitled "Die Architectur der Spongiosa" was often written in long, cumbersome sentences, with sometimes obscure meanings. We have taken considerable license in translating, rearranging punctuation, and condensing the material into modern terminology and style, while attempting to maintain the flavor of von Meyer's writing. We thank Dr. Per K. Amundson for the original translation; Drs. John Skedros and Richard Brand made additional suggestions. An accompanying biographical sketch of Hermann von Meyer is available at DOI 10.1007/s11999-011-2040-6.

Transfer of training-Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world.

Virtual reality (VR) can create safe, cost-effective, and engaging learning environments. It is commonly assumed that improvements in simulation fidelity lead to better learning outcomes. Some aspects of real environments, for example vestibular or haptic cues, are difficult to recreate in VR, but VR offers a wealth of opportunities to provide additional sensory cues in arbitrary modalities that provide task relevant information. The aim of this study was to investigate whether these cues improve user experience and learning outcomes, and, specifically, whether learning using augmented sensory cues translates into performance improvements in real environments. Participants were randomly allocated into three matched groups: Group 1 (control) was asked to perform a real tyre change only. The remaining two groups were trained in VR before performance was evaluated on the same, real tyre change task. Group 2 was trained using a conventional VR system, while Group 3 was trained in VR with augmented, task relevant, multisensory cues. Objective performance, time to completion and error number, subjective ratings of presence, perceived workload, and discomfort were recorded. The results show that both VR training paradigms improved performance for the real task. Providing additional, task-relevant cues during VR training resulted in higher objective performance during the real task. We propose a novel method to quantify the relative performance gains between training paradigms that estimates the relative gain in terms of training time. Systematic differences in subjective ratings that show comparable workload ratings, higher presence ratings and lower discomfort ratings, mirroring objective performance measures, were also observed. These findings further support the use of augmented multisensory cues in VR environments as an efficient method to enhance performance, user experience and, critically, the transfer of training from virtual to real environment scenarios.

Narrowing of the Audiovisual Temporal Binding Window Due To Perceptual Training Is Specific to High Visual Intensity Stimuli.

The temporal binding window (TBW), which reflects the range of temporal offsets in which audiovisual stimuli are combined to form a singular percept, can be reduced through training. Our research aimed to investigate whether training-induced reductions in TBW size transfer across stimulus intensities. A total of 32 observers performed simultaneity judgements at two visual intensities with a fixed auditory intensity, before and after receiving audiovisual TBW training at just one of these two intensities. We show that training individuals with a high visual intensity reduces the size of the TBW for bright stimuli, but this improvement did not transfer to dim stimuli. The reduction in TBW can be explained by shifts in decision criteria. Those trained with the dim visual stimuli, however, showed no reduction in TBW. Our main finding is that perceptual improvements following training are specific for high-intensity stimuli, potentially highlighting limitations of proposed TBW training procedures.

Reaction time facilitation for horizontally moving auditory-visual stimuli.

For moving targets, bimodal facilitation of reaction time has been observed for motion in the depth plane (C. Cappe, G. Thut, B. Romei, & M. M. Murray, 2009), but it is unclear whether analogous RT facilitation is observed for auditory-visual motion stimuli in the horizontal plane, as perception of horizontal motion relies on very different cues. Here we found that bimodal motion cues resulted in significant RT facilitation at threshold level, which could not be explained using an independent decisions model (race model). Bimodal facilitation was observed at suprathreshold levels when the RTs for suprathreshold unimodal stimuli were roughly equated, and significant RT gains were observed for direction-discrimination tasks with abrupt-onset motion stimuli and with motion preceded by a stationary phase. We found no speeded responses for bimodal signals when a motion signal in one modality was paired with a spatially co-localized stationary signal in the other modality, but faster response times could be explained by statistical facilitation when the motion signals traveled in opposite directions. These results strongly suggest that integration of motion cues led to the speeded bimodal responses. Finally, our results highlight the importance of matching the unimodal reaction times to obtain response facilitation for bimodal motion signals in the linear plane.

A colorimetric comparison of sunless with natural skin tan.

The main ingredient of sunless tanning products is dihydroxyacetone (DHA). DHA reacts with the protein and amino acid composition in the surface layers of the skin, producing melanoidins, which changes the skin colour, imitating natural skin tan caused by melanin. The purpose of this study was to characterise DHA-induced skin colour changes and to test whether we can predict the outcome of DHA application on skin tone changes. To assess the DHA-induced skin colour shift quantitatively, colorimetric and spectral measurements of the inner forearm were obtained before, four hours and 24 hours after application of a 7.5% concentration DHA gel in the experimental group (n = 100). In a control group (n = 60), the same measurements were obtained on both the inner forearm (infrequently sun-exposed) and the outer forearm (frequently sun-exposed); the difference between these two areas was defined as the naturally occurring tan. Skin colour shifts caused by DHA tanning and by natural tanning were compared in terms of lightness (L*), redness (a*) and yellowness (b*) in the standard CIELAB colour space. Naturalness of the DHA-induced skin tan was evaluated by comparing the trajectory of the chromaticity distribution in (L*, b*) space with that of naturally occurring tan. Twenty-four hours after DHA application, approximately 20% of the skin colour samples became excessively yellow, with chromaticities outside the natural range in (L*, b*) space. A principal component analysis was used to characterise the tanning pathway. Skin colour shifts induced by DHA were predicted by a multiple regression on the chromaticities and the skin properties. The model explained up to 49% of variance in colorimetric components with a median error of less than 2 ΔE. We conclude that the control of both the magnitude and the direction of the colour shift is a critical factor to achieve a natural appearance.

"Fake Tan" or "Fake News"?

We estimated Trump's skin colour from 70 internet images and also from the "twitter tan line" image (February 8, 2020; Twitter). We then compared the estimated skin colours with two existing data sets of skin colours: the range of skin tans that occur naturally in the Caucasian population and the range skin colours brought about by a sunless tan. We find that Trump's skin colour is close to the edge of the natural skin tan gamut and firmly within the gamut of a sunless skin tan. The skin colour above Trump's tan line is outside of the naturally occurring range of skin colours, even outside the skin tan of nonmelanized albinos. The latter finding is consistent with the hypothesis that part of the image may have been digitally distorted.

S-cone signals invisible to the motion system can improve motion extraction via grouping by color.

The purpose of this study was to test whether color-motion correlations carried by a pure color difference (S-cone component only) can be used to improve global motion extraction. We also examined the neural markers of color-motion correlation processing in event-related potentials. Color and motion information was dissociated using a two-colored random dot kinematogram, wherein coherent motion and motion noise differed from each other only in their S-cone component, with spatial and temporal parameters set so that global motion processing relied solely on a constant L-M component. Hence, when color and the local motion direction are correlated, more efficient segregation of coherent motion can only be brought about by the S-cone difference, and crucially, this S-cone component does not provide any effective input to a global motion mechanism but only changes the color appearance of the moving dots. The color contrasts (vector length in the S vs. L-M plane) of both the dots carrying coherent motion and the dots moving randomly were fixed at motion discrimination threshold to ensure equal effectiveness for motion extraction. In the behavioral experiment, participants were asked to discriminate between coherent and random motion, and d' was determined for three different conditions: uncorrelated, uncued correlated, and cued correlated. In the electroencephalographic experiment, participants discriminated direction of motion for uncued correlated and cued correlated conditions. Color-motion correlations were found to improve performance. Cueing a specific color also modulated the N1 component of the event-related potential, with sources in visual area middle temporal. We conclude that S-cone signals "invisible" to the motion system can influence the analysis by direction-selective motion mechanisms through grouping of local motion signals by color. This grouping mechanism must precede motion processing and is likely to be under attentional control.

Association Between Symptoms of Posttraumatic Stress Disorder and Signs of Temporomandibular Disorders in the General Population.

AIMS: To estimate the association between signs of temporomandibular disorders (TMD) and symptoms of posttraumatic stress disorder (PTSD) in a representative sample from the general population of northeastern Germany. METHODS: Signs of TMD were assessed with a clinical functional analysis that included palpation of the temporomandibular joints (TMJs) and masticatory muscles. PTSD was assessed with the PTSD module of the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, ed 4. The change-in-estimate method for binary logistic regression models was used to determine the final model and control for confounders. RESULTS: After the exclusion of subjects without prior traumatic events, the sample for joint pain consisted of 1,673 participants with a median age of 58.9 years (interquartile range 24.8), and the sample for muscle pain consisted of 1,689 participants with a median age of 59.1 years (interquartile range 24.8). Of these samples, 84 participants had pain on palpation of the TMJ, and 42 participants had pain on palpation of the masticatory muscles. Subjects having clinical PTSD (n = 62) had a 2.56-fold increase in joint pain (odds ratio [OR] = 2.56; 95% confidence interval [CI]: 1.14 to 5.71, P = .022) and a 3.86-fold increase (OR = 3.86; 95% CI: 1.51 to 9.85, P = .005) in muscle pain compared to subjects having no clinical PTSD. CONCLUSION: These results should encourage general practitioners and dentists to acknowledge the role of PTSD and traumatic events in the diagnosis and therapy of TMD, especially in a period of international migration and military foreign assignments.

Perceptual assessment of environmental stability modulates postural sway.

We actively maintain postural equilibrium in everyday life, and, although we are unaware of the underlying processing, there is increasing evidence for cortical involvement in this postural control. Converging evidence shows that we make appropriate use of 'postural anchors', for example static objects in the environment, to stabilise our posture. Visually evoked postural responses (VEPR) that are caused when we counteract the illusory perception of self-motion in space (vection) are modulated in the presence of postural anchors and therefore provide a convenient behavioural measure. The aim of this study is to evaluate the factors influencing visual appraisal of the suitability of postural anchors. We are specifically interested in the effect of perceived 'reality' in VR the expected 'stability' of visual anchors. To explore the effect of 'reality' we introduced an accommodation-vergence conflict. We show that VEPR are appropriately modulated only when virtual visual 'anchors' are rendered such that vergence and accommodation cues are consistent. In a second experiment we directly test whether cognitive assessment of the likely stability of real perceptual anchors (we contrast a 'teapot on a stand' and a 'helium balloon') affects VEPR. We show that the perceived positional stability of environmental anchors modulate postural responses. Our results confirm previous findings showing that postural sway is modulated by the configuration of the environment and further show that an assessment of the stability and reality of the environment plays an important role in this process. On this basis we propose design guidelines for VR systems, in particular we argue that accommodation-vergence conflicts should be minimised and that high quality motion tracking and rendering are essential for high fidelity VR.

The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment.

Objective and subjective measures of performance in virtual reality environments increase as more sensory cues are delivered and as simulation fidelity increases. Some cues (colour or sound) are easier to present than others (object weight, vestibular cues) so that substitute cues can be used to enhance informational content in a simulation at the expense of simulation fidelity. This study evaluates how substituting cues in one modality by alternative cues in another modality affects subjective and objective performance measures in a highly immersive virtual reality environment. Participants performed a wheel change in a virtual reality (VR) environment. Auditory, haptic and visual cues, signalling critical events in the simulation, were manipulated in a factorial design. Subjective ratings were recorded via questionnaires. The time taken to complete the task was used as an objective performance measure. The results show that participants performed best and felt an increased sense of immersion and involvement, collectively referred to as 'presence', when substitute multimodal sensory feedback was provided. Significant main effects of audio and tactile cues on task performance and on participants' subjective ratings were found. A significant negative relationship was found between the objective (overall completion times) and subjective (ratings of presence) performance measures. We conclude that increasing informational content, even if it disrupts fidelity, enhances performance and user's overall experience. On this basis we advocate the use of substitute cues in VR environments as an efficient method to enhance performance and user experience.