Mixed Reality Interaction and Presentation Techniques for Medical Visualisations.

Mixed, Augmented and Virtual reality technologies are burgeoning with new applications and use cases appearing rapidly. This chapter provides a brief overview of the fundamental display presentation methods; head-worn, hand-held and projector-based displays. We present a summary of visualisation methods that employ these technologies in the medical domain with a diverse range of examples presented including diagnostic and exploration, intervention and clinical, interaction and gestures, and education.

Bogus visual feedback alters onset of movement-evoked pain in people with neck pain.

Pain is a protective perceptual response shaped by contextual, psychological, and sensory inputs that suggest danger to the body. Sensory cues suggesting that a body part is moving toward a painful position may credibly signal the threat and thereby modulate pain. In this experiment, we used virtual reality to investigate whether manipulating visual proprioceptive cues could alter movement-evoked pain in 24 people with neck pain. We hypothesized that pain would occur at a lesser degree of head rotation when visual feedback overstated true rotation and at a greater degree of rotation when visual feedback understated true rotation. Our hypothesis was clearly supported: When vision overstated the amount of rotation, pain occurred at 7% less rotation than under conditions of accurate visual feedback, and when vision understated rotation, pain occurred at 6% greater rotation than under conditions of accurate visual feedback. We concluded that visual-proprioceptive information modulated the threshold for movement-evoked pain, which suggests that stimuli that become associated with pain can themselves trigger pain.

Smart Pipette: Elevating Laboratory Performance with Tactile Authenticity and Real-Time Feedback.

Mastering the correct use of laboratory equipment is a fundamental skill for undergraduate science students involved in laboratory-based training. However, hands-on laboratory time is often limited, and remote students may struggle as their absence from the physical lab limits their skill development. An air-displacement micropipette was selected for our initial investigation, as accuracy and correct technique are essential in generating reliable assay data. Handling small liquid volumes demands hand dexterity and practice to achieve proficiency. This research assesses the importance of tactile authenticity during training by faithfully replicating the micropipette's key physical and operational characteristics. We developed a custom haptic training approach called 'Smart Pipette' which promotes accurate operation and enhances laboratory dexterity training. A comparative user study with 34 participants evaluated the effectiveness of the Smart Pipette custom haptic device against training with off-the-shelf hardware, specifically the Quest VR hand controller, which was chosen because it is held mid-air similar to a laboratory micropipette. Both training conditions are integrated with the same self-paced virtual simulation displayed on a computer screen, offering clear video instructions and realtime guidance. Results demonstrated that participants trained with the Smart Pipette custom haptic exhibited increased accuracy and precision while making fewer errors than those trained with off-the-shelf hardware. The Smart Pipette and the Quest VR controller had no significant differences in cognitive load and system usability scores. Tactile authentic interaction devices address challenges faced by online learners, while their applicability extends to traditional classrooms, where real-time feedback significantly enhances overall training performance outcomes.

Personalised virtual reality in palliative care: clinically meaningful symptom improvement for some.

OBJECTIVES: This study examined the effects of virtual reality (VR) among palliative care patients at an acute ward. Objectives included evaluating VR therapy benefits across three sessions, assessing its differential impact on emotional versus physical symptoms and determining the proportion of patients experiencing clinically meaningful improvements after each session. METHODS: A mixed-methods design was employed. Sixteen palliative inpatients completed three personalised 20 min VR sessions. Symptom burden was assessed using the Edmonton Symptom Assessment Scale-Revised and quality of life with the Functional Assessment of Chronic Illness Therapy (FACIT-Pal-14). Standardised criteria assessed clinically meaningful changes. Quantitative data were analysed using linear mixed models. RESULTS: Quality of life improved significantly pre-VR to post-VR with a large effect size (Cohen's d: 0.98). Total symptom burden decreased after 20 min VR sessions (Cohen's d: 0.75), with similar effect sizes for emotional (Cohen's d: 0.67) and physical symptoms (Cohen's d: 0.63). Over 50% of patients experienced clinically meaningful improvements per session, though substantial individual variability occurred. CONCLUSIONS: This study reveals the nuanced efficacy of personalised VR therapy in palliative care, with over half of the patients experiencing meaningful benefits in emotional and physical symptoms. The marked variability in responses underscores the need for realistic expectations when implementing VR therapy.

Adaptive Reset Techniques for Haptic Retargeted Interaction.

This article presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors.

What I see and what I feel: the influence of deceptive visual cues and interoceptive accuracy on affective valence and sense of effort during virtual reality cycling.

Background: How we feel during exercise is influenced by exteroceptive (e.g., vision) and interoceptive (i.e., internal body signals) sensory information, and by our prior experiences and expectations. Deceptive visual cues about one's performance during exercise can increase work rate, without negatively impacting affective valence (good/bad responses) or perceived exertion. However, what is less understood is whether the perception of the exercise experience itself can be shifted, if work rate is held constant. Here we aimed to investigate whether deceptive vision-via illusory hills in a virtual reality (VR) cycling experience-alters affective valence and perceived exertion when physical effort is controlled. We also evaluated whether the accuracy with which one detects interoceptive cues influences the extent to which deceptive visual information can shift exercise experiences. Methods: A total of 20 participants (10 female; 30.2 ± 11.2 yrs) completed three VR cycling conditions each of 10-min duration, in a randomised, counterbalanced order. Pedal resistance/cadence were individualised (to exercise intensity around ventilatory threshold) and held constant across conditions; only visual cues varied. Two conditions provided deceptive visual cues about the terrain (illusory uphill, illusory downhill; resistance did not change); one condition provided accurate visual cues (flat terrain). Ratings of affective valence (Feeling Scale) and of perceived exertion (Borg's RPE) were obtained at standardised timepoints in each VR condition. Interoceptive accuracy was measured via a heartbeat detection test. Results: Linear mixed effects models revealed that deceptive visual cues altered affective valence (f2 = 0.0198). Relative to flat terrain, illusory downhill reduced affective valence (Est = -0.21, p = 0.003), but illusory uphill did not significantly improve affective valence (Est = 0.107, p = 0.14). Deceptive visual cues altered perceived exertion, and this was moderated by the level of interoceptive accuracy (Condition-Interoception interaction, p = 0.00000024, f2 = 0.0307). Higher levels of interoceptive accuracy resulted in higher perceived exertion in the illusory downhill condition (vs flat), while lower interoceptive accuracy resulted in lower perceived exertion in both illusory hill conditions (vs flat) and shifts of greater magnitude. Conclusions: Deceptive visual cues influence perceptual responses during exercise when physical effort does not vary, and for perceived exertion, the weighting given to visual exteroceptive cues is determined by accuracy with which interoceptive cues are detected. Contrary to our hypotheses, deceptive visual cues did not improve affective valence. Our findings suggest that those with lower levels of interoceptive accuracy experience most benefit from deceptive visual cues, providing preliminary insight into individualised exercise prescription to promote positive (and avoid negative) exercise experiences.

Your best day: An interactive app to translate how time reallocations within a 24-hour day are associated with health measures.

Reallocations of time between daily activities such as sleep, sedentary behavior and physical activity are differentially associated with markers of physical, mental and social health. An individual's most desirable allocation of time may differ depending on which outcomes they value most, with these outcomes potentially competing with each other for reallocations. We aimed to develop an interactive app that translates how self-selected time reallocations are associated with multiple health measures. We used data from the Australian Child Health CheckPoint study (n = 1685, 48% female, 11-12 y), with time spent in daily activities derived from a validated 24-h recall instrument, %body fat from bioelectric impedance, psychosocial health from the Pediatric Quality of Life Inventory and academic performance (writing) from national standardized tests. We created a user-interface to the compositional isotemporal substitution model with interactive sliders that can be manipulated to self-select time reallocations between activities. The time-use composition was significantly associated with body fat percentage (F = 2.66, P < .001), psychosocial health (F = 4.02, P < .001), and academic performance (F = 2.76, P < .001). Dragging the sliders on the app shows how self-selected time reallocations are associated with the health measures. For example, reallocating 60 minutes from screen time to physical activity was associated with -0.8 [95% CI -1.0 to -0.5] %body fat, +1.9 [1.4 to 2.5] psychosocial score and +4.5 [1.8 to 7.2] academic performance. Our app allows the health associations of time reallocations to be compared against each other. Interactive interfaces provide flexibility in selecting which time reallocations to investigate, and may transform how research findings are disseminated.

Characterising activity and diet compositions for dementia prevention: protocol for the ACTIVate prospective longitudinal cohort study.

INTRODUCTION: Approximately 40% of late-life dementia may be prevented by addressing modifiable risk factors, including physical activity and diet. Yet, it is currently unknown how multiple lifestyle factors interact to influence cognition. The ACTIVate Study aims to (1) explore associations between 24-hour time-use and diet compositions with changes in cognition and brain function; and (2) identify duration of time-use behaviours and the dietary compositions to optimise cognition and brain function. METHODS AND ANALYSIS: This 3-year prospective longitudinal cohort study will recruit 448 adults aged 60-70 years across Adelaide and Newcastle, Australia. Time-use data will be collected through wrist-worn activity monitors and the Multimedia Activity Recall for Children and Adults. Dietary intake will be assessed using the Australian Eating Survey food frequency questionnaire. The primary outcome will be cognitive function, assessed using the Addenbrooke's Cognitive Examination-III. Secondary outcomes include structural and functional brain measures using MRI, cerebral arterial pulse measured with diffuse optical tomography, neuroplasticity using simultaneous transcranial magnetic stimulation and electroencephalography, and electrophysiological markers of cognitive control using event-related potential and time frequency analyses. Compositional data analysis, testing for interactions between time point and compositions, will assess longitudinal associations between dependent (cognition, brain function) and independent (time-use and diet compositions) variables. CONCLUSIONS: The ACTIVate Study will be the first to examine associations between time-use and diet compositions, cognition and brain function. Our findings will inform new avenues for multidomain interventions that may more effectively account for the co-dependence between activity and diet behaviours for dementia prevention. ETHICS AND DISSEMINATION: Ethics approval has been obtained from the University of South Australia's Human Research Ethics committee (202639). Findings will be disseminated through peer-reviewed manuscripts, conference presentations, targeted media releases and community engagement events. TRIAL REGISTRATION NUMBER: Australia New Zealand Clinical Trials Registry (ACTRN12619001659190).

Illusion-enhanced Virtual Reality Exercise for Neck Pain: A Replicated Single Case Series.

OBJECTIVES: Body illusions have shown promise in treating some chronic pain conditions. We hypothesized that neck exercises performed in virtual reality (VR) with visual feedback of rotation amplified would reduce persistent neck pain. METHODS: In a multiple-baseline replicated single case series, 8 blinded individuals with persistent neck pain completed a 4-phase intervention (initial n=12, 4 dropouts): (1) "baseline"; (2) "VR" during which participants performed rotation exercises in VR with no manipulation of visual feedback; (3) "VR enhanced" during which identical exercises were performed but visual feedback overstated the range of motion being performed; (4) "follow-up." Primary outcomes were twice-daily measures of pain-free range of motion and pain intensity. During the baseline and follow-up phases, measures were taken but no intervention took place. RESULTS: No differences in primary outcomes were found between VR and baseline, VR enhanced and VR, or VR enhanced and follow-up. DISCUSSION: Our hypothesis, that neck exercises performed in VR with visual feedback of rotation amplified, would reduce persistent neck pain was not supported. Possible explanations and future directions are discussed.

Using visuo-kinetic virtual reality to induce illusory spinal movement: the MoOVi Illusion.

BACKGROUND: Illusions that alter perception of the body provide novel opportunities to target brain-based contributions to problems such as persistent pain. One example of this, mirror therapy, uses vision to augment perceived movement of a painful limb to treat pain. Since mirrors can't be used to induce augmented neck or other spinal movement, we aimed to test whether such an illusion could be achieved using virtual reality, in advance of testing its potential therapeutic benefit. We hypothesised that perceived head rotation would depend on visually suggested movement. METHOD: In a within-subjects repeated measures experiment, 24 healthy volunteers performed neck movements to 50o of rotation, while a virtual reality system delivered corresponding visual feedback that was offset by a factor of 50%-200%-the Motor Offset Visual Illusion (MoOVi)-thus simulating more or less movement than that actually occurring. At 50o of real-world head rotation, participants pointed in the direction that they perceived they were facing. The discrepancy between actual and perceived direction was measured and compared between conditions. The impact of including multisensory (auditory and visual) feedback, the presence of a virtual body reference, and the use of 360o immersive virtual reality with and without three-dimensional properties, was also investigated. RESULTS: Perception of head movement was dependent on visual-kinaesthetic feedback (p = 0.001, partial eta squared = 0.17). That is, altered visual feedback caused a kinaesthetic drift in the direction of the visually suggested movement. The magnitude of the drift was not moderated by secondary variables such as the addition of illusory auditory feedback, the presence of a virtual body reference, or three-dimensionality of the scene. DISCUSSION: Virtual reality can be used to augment perceived movement and body position, such that one can perform a small movement, yet perceive a large one. The MoOVi technique tested here has clear potential for assessment and therapy of people with spinal pain.

Cognitive Cost of Using Augmented Reality Displays.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

Neck Pain and Proprioception Revisited Using the Proprioception Incongruence Detection Test.

BACKGROUND: Proprioceptive imprecision is believed to contribute to persistent pain. Detecting imprecision in order to study or treat it remains challenging given the limitations of current tests. OBJECTIVES: The aim of this study was to determine whether proprioceptive imprecision could be detected in people with neck pain by testing their ability to identify incongruence between true head motion and a false visual reference using the Proprioception Incongruence Detection (PID) Test. DESIGN: A cross-sectional study was conducted. METHODS: Twenty-four people with neck pain and 24 matched controls repeatedly rotated to specific markers within a virtual world and indicated if their true head rotation was more or less than the rotation suggested by the visual feedback. Visual feedback was manipulated at 6 corrections, ranging from 60% of true movement to 140% of true movement. A standard repositioning error (RPE) test as undertaken for comparison. RESULTS: Healthy controls were better able to detect incongruence between vision and true head rotation (X̅=75.6%, SD=8.5%) than people with neck pain were (X̅=69.6%, SD=12.7%). The RPE test scores were not different between groups. The PID Test score related to self-reported pain intensity but did not relate to RPE test score. LIMITATIONS: Causality cannot be established from this cross-sectional study, and further work refining the PID Test is needed for it to offer clinical utility. CONCLUSIONS: Proprioceptive precision for neck movement appears worse in people with neck pain than in those without neck pain, and the extent of the deficit appears to be related to usual pain severity. The PID Test appears to be a more sensitive test than the RPE test and is likely to be useful for assessment of proprioceptive function in research and clinical settings.

Augmented Reality as a Countermeasure for Sleep Deprivation.

Sleep deprivation is known to have serious deleterious effects on executive functioning and job performance. Augmented reality has an ability to place pertinent information at the fore, guiding visual focus and reducing instructional complexity. This paper presents a study to explore how spatial augmented reality instructions impact procedural task performance on sleep deprived users. The user study was conducted to examine performance on a procedural task at six time points over the course of a night of total sleep deprivation. Tasks were provided either by spatial augmented reality-based projections or on an adjacent monitor. The results indicate that participant errors significantly increased with the monitor condition when sleep deprived. The augmented reality condition exhibited a positive influence with participant errors and completion time having no significant increase when sleep deprived. The results of our study show that spatial augmented reality is an effective sleep deprivation countermeasure under laboratory conditions.

Geometrically-correct projection-based texture mapping onto a deformable object.

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object’s surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.

Spatial user interfaces for large-scale projector-based augmented reality.

Spatial augmented reality applies the concepts of spatial user interfaces to large-scale, projector-based augmented reality. Such virtual environments have interesting characteristics. They deal with large physical objects, the projection surfaces are nonplanar, the physical objects provide natural passive haptic feedback, and the systems naturally support collaboration between users. The article describes how these features affect the design of spatial user interfaces for these environments and explores promising research directions and application domains.