Mixed reality-supported near-infrared photoimmunotherapy for oropharyngeal cancer: a case report.

Introduction and importance: Near-infrared photoimmunotherapy (NIR-PIT, Alluminox) uses an antibody-photoabsorber conjugate and light excitation, requiring precise illumination. Mixed reality (MR) technology can enhance medical procedures through advanced visualization and planning. Case presentation: An 86-year-old man with recurrent oropharyngeal cancer and right cervical metastasis received NIR-PIT. Three-dimensional models from computed tomography (CT) and FDG-PET/CT images were used as holograms on a head-mounted display (HMD) for precise light targeting. Clinical discussion: HMD-MR technology was utilized for preoperative simulation and guided ideal light direction during surgery. This improved the effectiveness of NIR-PIT. Conclusion: Three months post-treatment, no residual lesion was observed, demonstrating the utility of HMD-MR technology in optimizing NIR-PIT outcomes.

Virtual reality perceptual training can improve the temporal discrimination ability of swinging during softball batting.

Perception and action uncoupling in fastball sports anticipatory training is often criticized. Nevertheless, perception-only training offers distinct advantages, such as flexibility concerning time, place, and injury limitations. Therefore, the effectiveness of this training approach warrants evaluation. This study developed a virtual reality (VR) training system based on the idea that the two visual pathways in the brain are associated with visual information attributes, rather than perception or action. A key feature of this study's perception-only training was its presentation of not only the opponent's kinematics but also the ball's flight information (the attributes that guide hitting) to train the visual system necessary for real situations. Seventeen female softball batters were assigned to two groups: a training group (N = 9) and a control group (N = 8). Only the training group underwent the VR anticipatory skill training to discriminate the different ball speeds. Both groups completed a perception test and an action test on the VR during the pre- and post-training periods. The perception test assessed response accuracy (RA) in discriminating ball speed, and the action test assessed the temporal difference in swing onset (delta onset). Comparison of these two outcome variables between groups revealed that perception-only training improved both perceptual and action responses. This study demonstrated the effectiveness of perception-only training and emphasized the need for its thoughtful utilization and design.

Developing a Virtual Reality Simulation System for Preoperative Planning of Robotic-Assisted Thoracic Surgery.

Background. Robotic-assisted thoracic surgery (RATS) is now standard for lung cancer treatment, offering advantages over traditional methods. However, RATS's minimally invasive approach poses challenges like limited visibility and tactile feedback, affecting surgeons' navigation through com-plex anatomy. To enhance preoperative familiarization with patient-specific anatomy, we devel-oped a virtual reality (VR) surgical navigation system. Using head-mounted displays (HMDs), this system provides a comprehensive, interactive view of the patient's anatomy pre-surgery, aiming to improve preoperative simulation and intraoperative navigation. Methods. We integrated 3D data from preoperative CT scans into Perspectus VR Education software, displayed via HMDs for in-teractive 3D reconstruction of pulmonary structures. This detailed visualization aids in tailored preoperative resection simulations. During RATS, surgeons access these 3D images through Tile-ProTM multi-display for real-time guidance. Results. The VR system enabled precise visualization of pulmonary structures and lesion relations, enhancing surgical safety and accuracy. The HMDs offered true 3D interaction with patient data, facilitating surgical planning. Conclusions. VR sim-ulation with HMDs, akin to a robotic 3D viewer, offers a novel approach to developing robotic surgical skills. Integrated with routine imaging, it improves preoperative planning, safety, and accuracy of anatomical resections. This technology particularly aids in lesion identification in RATS, optimizing surgical outcomes.

The effects of head mounted weight on comfort for helmets and headsets, with a definition of "comfortable wear time".

BACKGROUND: There are difficult tradeoffs when designing head-mounted equipment such as helmets, lights, cameras, or virtual or augmented reality displays. Increased functionality and battery life adds weight, which in turn reduces comfort. A successful product must balance both comfort and functionality to achieve its product engagement goals. OBJECTIVE: This study defines "comfortable wear time" as a new metric, and applies it to the domain of headsets in determining the relationship between headset weight and comfort. METHODS: Sixteen study participants wore four otherwise identical headsets weighted between 500g-600 g for up to two hours each in an office environment. If participants experienced more than "mild discomfort" (>3 on an NRS-11 discomfort scale), the trial ended early, and the comfortable wear time was recorded. Intensity and location of discomfort was rated at trial conclusion, and qualitative feedback collected. RESULTS: Higher weights were associated with shorter comfortable wear times. Not everyone could wear even the lightest headset (500 g) for the full two hours. Qualitatively, discomfort took many forms beyond the expected neck fatigue or contact pressure and included symptoms commonly associated with motion sickness, such as headache and dizziness. Finally, there were pronounced gender differences with females experiencing more severe discomfort with earlier onset. CONCLUSION: Heavier headsets were less comfortable for the lower quartile of participants -yielding an average of 11 fewer minutes of comfortable wear time per 33 g of weight added. Understanding the discomfort costs from adding weight empowers product teams to find the correct balance to meet their product engagement targets.

The development of new remote technologies in disaster medicine education: A scoping review.

Background: Remote teaching and online learning have significantly changed the responsiveness and accessibility after the COVID-19 pandemic. Disaster medicine (DM) has recently gained prominence as a critical issue due to the high frequency of worldwide disasters, especially in 2021. The new artificial intelligence (AI)-enhanced technologies and concepts have recently progressed in DM education. Objectives: The aim of this article is to familiarize the reader with the remote technologies that have been developed and used in DM education over the past 20 years. Literature scoping reviews: Mobile edge computing (MEC), unmanned aerial vehicles (UAVs)/drones, deep learning (DL), and visual reality stimulation, e.g., head-mounted display (HMD), are selected as promising and inspiring designs in DM education. Methods: We performed a comprehensive review of the literature on the remote technologies applied in DM pedagogy for medical, nursing, and social work, as well as other health discipline students, e.g., paramedics. Databases including PubMed (MEDLINE), ISI Web of Science (WOS), EBSCO (EBSCO Essentials), Embase (EMB), and Scopus were used. The sourced results were recorded in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart and followed in accordance with the PRISMA extension Scoping Review checklist. We included peer-reviewed articles, Epubs (electronic publications such as databases), and proceedings written in English. VOSviewer for related keywords extracted from review articles presented as a tabular summary to demonstrate their occurrence and connections among these DM education articles from 2000 to 2022. Results: A total of 1,080 research articles on remote technologies in DM were initially reviewed. After exclusion, 64 articles were included in our review. Emergency remote teaching/learning education, remote learning, online learning/teaching, and blended learning are the most frequently used keywords. As new remote technologies used in emergencies become more advanced, DM pedagogy is facing more complex problems. Discussions: Artificial intelligence-enhanced remote technologies promote learning incentives for medical undergraduate students or graduate professionals, but the efficacy of learning quality remains uncertain. More blended AI-modulating pedagogies in DM education could be increasingly important in the future. More sophisticated evaluation and assessment are needed to implement carefully considered designs for effective DM education.

Use of a Novel Three-dimensional Head-mounted Digital Visualization Platform in Corneal Endothelial Transplantation.

INTRODUCTION: To report the first endothelial keratoplasty procedures performed using a 3D digital head-mounted ophthalmic exoscope. METHODS: Three patients underwent Descemet stripping automated endothelial keratoplasty (DSAEK) using a 3D digital ophthalmic exoscope (Beyeonics One, Beyeonics Vision, Haifa, Israel) at the Tel Aviv Sourasky Medical Center, Tel Aviv, Israel. RESULTS: All procedures were uneventful, leading to resolution of corneal edema and vision improvement. Surgeons reported excellent visualization and minimal lag, almost negligible, with the benefits of improved ergonomics and the use of head gestures to control zoom, focus, brightness, and panning. There were no postoperative complications. CONCLUSION: The new 3D digital ophthalmic exoscope system can be successfully used in DSAEK surgery with potential advantages in ergonomics, picture quality, and image control. Further studies can compare this system with either standard operating microscopes or 3D heads-up display systems.

Navigation in Contour-Drawn Scenes Using Augmented Reality.

The visual system can recover 3D information from many different types of visual information, e.g., contour-drawings. How well can people navigate in a real dynamic environment with contour-drawings? This question was addressed by developing an AR-device that could show a contour-drawing of a real scene in an immersive manner and by conducting an observational field study in which the two authors navigated in real environments wearing this AR-device. The navigation with contour-drawings was difficult in natural scenes but easy in urban scenes. This suggests that the visual information from natural and urban environments is sufficiently different and our visual system can accommodate to this difference of the visual information in different environments.

A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality.

Over the years, the various mediums available for storytelling have progressively expanded, from spoken to written word, then to film, and now to Virtual Reality (VR) and Augmented Reality (AR). In 2016, the cutting-edge Head-Mounted Display (HMD) AR Microsoft HoloLens was released. However, though it has been several years, the quality of the user experience with narration using HMD-based AR technology has been rarely discussed. The present study explored interactive narrative in HMD-based AR regarding different user interfaces and their influence on users' presence, narrative engagement and reflection. Inspired by an existing exhibition at the National Holocaust Centre and Museum in the U.K., a HoloLens narrative application, entitled The AR Journey, was developed by the authors using two different interaction methods, Natural User Interface (NUI) and Graphical User Interface (GUI), which were used to perform an empirical study. As revealed from the results of the between-subject design experiment, NUI exhibited statistically significant advantages in creating presence for users without 3D Role Playing Game (RPG) experience, and GUI was superior in creating presence and increasing narrative engagement for users with 3D RPG experience. As indicated by the results of the interviews, the overall narrative experience in HMD-based AR was acceptable, and the branching narrative design was engaging. However, HoloLens hardware issues, as well as virtuality and reality mismatch, adversely affected user experience. Design guidelines were proposed according to the qualitative results. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s11042-021-11723-0.

Developing a virtual reality simulation system for preoperative planning of thoracoscopic thoracic surgery.

BACKGROUND: Video-assisted thoracoscopic surgery (VATS) has become a standard approach for the treatment of lung cancer. However, its minimally invasive nature limits the field of view and reduces tactile feedback. These limitations make it vital that surgeons thoroughly familiarize themselves with the patient's anatomy preoperatively. We have developed a virtual reality (VR) surgical navigation system using head-mounted displays (HMD). The aim of this study was to investigate the potential utility of this VR simulation system in both preoperative planning and intraoperative assistance, including support during thoracoscopic sublobar resection. METHODS: Three-dimensional (3D) polygon data derived from preoperative computed tomography data was loaded into BananaVision software developed at Colorado State University and displayed on an HMD. An interactive 3D reconstruction image was created, in which all the pulmonary structures could be individually imaged. Preoperative resection simulations were performed with patient-individualized reconstructed 3D images. RESULTS: The 3D anatomic structure of pulmonary vessels and a clear vision into the space between the lesion and adjacent tissues were successfully appreciated during preoperative simulation. Surgeons could easily evaluate the real patient's anatomy in preoperative simulations to improve the accuracy and safety of actual surgery. The VR software and HMD allowed surgeons to visualize and interact with real patient data in true 3D providing a unique perspective. CONCLUSIONS: This initial experience suggests that a VR simulation with HMD facilitated preoperative simulation. Routine imaging modalities combined with VR systems could substantially improve preoperative planning and contribute to the safety and accuracy of anatomic resection.

Analysis of Street-Crossing Behavior: Comparing a CAVE Simulator and a Head-Mounted Display among Younger and Older Adults.

Interactive pedestrian simulators have become a valuable research tool for investigating street-crossing behavior and developing solutions for improving pedestrian safety. There are two main kinds of pedestrian simulators: one uses a technology based on rear-projection screens (Cave Automatic Virtual Environment, or CAVE), the other a head-mounted display (HMD). These devices are used indiscriminately, regardless of the research objective, and it is not yet known whether they are equally effective for studying street crossing. The present study was aimed at comparing the street crossing behavior and subjective evaluations of younger and older adult pedestrians when they are using a CAVE-like or HMD-based (HTC Vive Pro) pedestrian simulator. Thirty younger adults and 25 older adults performed 36 street-crossing trials (combining different speeds, two-way traffic conditions, and gap sizes) on each of the two types of simulators. The results indicated that participants in the HMD condition crossed the street significantly more often (58.6 %) than in the CAVE condition (42.44%) and had shorter safety margins. The most striking difference pertained to crossing initiation, which occurred considerably earlier (1.78 s) in the HMD condition than in the CAVE condition. Synchronization of crossing initiation with oncoming traffic was not as good in the CAVE condition because visual information in front of the pedestrian was missing due to the absence of ground projection. In both simulators, older adults caused more collisions than did younger ones, had shorter safety margins, and a slower crossing speed. Hence, the HMD reproduced classical age-related differences in most street-crossing behaviors already found on the CAVE. Usually observed speed effects were also found for both simulators. Neither cybersickness nor any adverse effects on stereoacuity or postural balance were found for either simulator. The HMD produced a higher level of presence and preference than the CAVE did. These findings provide evidence that HMDs have a clear potential for studying pedestrian behaviour.

Eye-Tracking for Clinical Ophthalmology with Virtual Reality (VR): A Case Study of the HTC Vive Pro Eye's Usability.

BACKGROUND: A case study is proposed to empirically test and discuss the eye-tracking status-quo hardware capabilities and limitations of an off-the-shelf virtual reality (VR) headset with embedded eye-tracking for at-home ready-to-go online usability in ophthalmology applications. METHODS: The eye-tracking status-quo data quality of the HTC Vive Pro Eye is investigated with novel testing specific to objective online VR perimetry. Testing was done across a wide visual field of the head-mounted-display's (HMD) screen and in two different moving conditions. A new automatic and low-cost Raspberry Pi system is introduced for VR temporal precision testing for assessing the usability of the HTC Vive Pro Eye as an online assistance tool for visual loss. RESULTS: The target position on the screen and head movement evidenced limitations of the eye-tracker capabilities as a perimetry assessment tool. Temporal precision testing showed the system's latency of 58.1 milliseconds (ms), evidencing its good potential usage as a ready-to-go online assistance tool for visual loss. CONCLUSIONS: The test of the eye-tracking data quality provides novel analysis useful for testing upcoming VR headsets with embedded eye-tracking and opens discussion regarding expanding future introduction of these HMDs into patients' homes for low-vision clinical usability.

Spatial presence depends on 'coupling' between body sway and visual motion presented on head-mounted displays (HMDs).

This study investigated the effects of simulating self-motion via a head-mounted display (HMD) on standing postural sway and spatial presence. Standing HMD users viewed simulated oscillatory self-motion in depth. On a particular trial, this naso-occipital visual oscillation had one of four different amplitudes (either 4, 8, 12 or 16 m peak-to-peak) and one of four different frequencies (either 0.125, 0.25, 0.5 or 1 Hz). We found that simulated high amplitude self-oscillation (approximately 16 m peak-to-peak) at either 0.25 Hz or 0.5 Hz: 1) generated the strongest effects on postural sway; and 2) made participants feel more spatially present in the virtual environment. Our findings provide insight into the parameters of simulated self-motion that generate the strongest postural responses within virtual environments. These postural constraints have valuable implications for improving our understanding of sensory processes underlying the ergonomic experience of virtual environments simulated using HMDs.

Triggering Postural Movements With Virtual Reality Technology in Healthy Young and Older Adults: A Cross-Sectional Validation Study for Early Dementia Screening.

With the ultimate aim of early diagnosis of dementia, a new body balance assessment system with integrated head-mounted display-based virtual reality (VR) has been developed. We hypothesized that people would sway more in anterior-posterior (AP) direction when they were exposed to a VR environment where we intentionally provoked movements in forward and backward directions. A total of 14 healthy older adults (OA) (73.14±4.26 years) and 15 healthy young adults (YA) (24.93±1.49 years) were assessed for group differences in sway behavior. Body sway speed in 22 different conditions with and without VR environments was analyzed. Significant differences and large effect sizes were observed in AP sway under the VR environments (OA with P < 0.02; effect size> 0.61, YA with P < 0.003; effect size> 0.72) compared to the baseline condition without the VR environments. In addition, significant differences were found between the two groups in AP sway in all test conditions (P < 0.01). Our study shows that a VR environment can trigger body sway in an expected direction, which may indicate that it is possible to enhance the sensitivity of balance assessment by integrating immersive VR environments. The result of this study warrants a cross-sectional study in which OA diagnosed with and without dementia are compared on their sway behavior.

Assessing Saccadic Eye Movements With Head-Mounted Display Virtual Reality Technology.

As our society is ageing globally, neurodegenerative disorders are becoming a relevant issue. Assessment of saccadic eye movement could provide objective values to help to understand the symptoms of disorders. HTC Corporation launched a new virtual reality (VR) headset, VIVE Pro Eye, implementing an infrared-based eye tracking technique together with VR technology. The purpose of this study is to evaluate whether the device can be used as an assessment tool of saccadic eye movement and to investigate the technical features of eye tracking. We developed a measurement system of saccadic eye movement with a simple VR environment on Unity VR design platform, following an internationally proposed standard saccade measurement protocol. We then measured the saccadic eye movement of seven healthy young adults to analyze the oculo-metrics of latency, peak velocity, and error rate of pro- and anti-saccade tasks: 120 trials in each task. We calculated these parameters based on the saccade detection algorithm that we have developed following previous studies. Consequently, our results revealed latency of 220.40 ± 43.16 ms, peak velocity of 357.90 ± 111.99°/s, and error rate of 0.24 ± 0.41% for the pro-saccade task, and latency of 343.35 ± 76.42 ms, peak velocity of 318.79 ± 116.69°/s, and error rate of 0.66 ± 0.76% for the anti-saccade task. In addition, we observed pupil diameter of 4.30 ± 1.15 mm (left eye) and 4.29 ± 1.08 mm (right eye) for the pro-saccade task, and of 4.21 ± 1.04 mm (left eye) and 4.22 ± 0.97 mm (right eye) for the anti-saccade task. Comparing between the descriptive statistics of previous studies and our results suggests that VIVE Pro Eye can function as an assessment tool of saccadic eye movement since our results are in the range of or close to the results of previous studies. Nonetheless, we found technical limitations especially about time-related measurement parameters. Further improvements in software and hardware of the device and measurement protocol, and more measurements with diverse age-groups and people with different health conditions are warranted to enhance the whole assessment system of saccadic eye movement.

Postural responses to sinusoidal modulations of viewpoint position in a virtual environment.

Visual self-motion information is known to contribute to postural control, but it is unclear precisely which aspects of visual motion information drive changes in posture. We report here results for standing humans which suggest that there is a speed of movement threshold that must be exceeded by a visual stimulus if a posture response is to be generated. We use signal-to-noise ratio (SNR) methods to measure the strength of steady-state visually evoked posture responses (SSVEPRs) to sinusoidal modulations of visual viewpoint position in a virtual environment (VE). Using threshold estimates found from data which show how posture responses depend on visual stimulus amplitude, we show that the sensitivity of the visuo-postural response system increases with the temporal frequency at which the position of one's viewpoint is modulated. We show further that there is a speed of movement threshold, on average 1.85 cm/s, which must be exceeded by a left-right modulation of viewpoint position if a posture response is to be generated. A comparison of visual stimulus visibility to posture response thresholds suggests that one tends to not make postural responses to visual stimuli that are unseen. Finally, we found small correlations between motion sickness in these experiments and both the time spent in the VE and the frequency of viewpoint movement.

The Influence of Virtual Reality Head-Mounted Displays on Balance Outcomes and Training Paradigms: A Systematic Review.

Background: Falls are the leading causes of (non)fatal injuries in older adults. Recent research has developed interventions that aim to improve balance in older adults using virtual reality (VR). Purpose: We aimed to investigate the validity, reliability, safety, feasibility, and efficacy of head mounted display (HMD) systems for assessing and training balance in older adults. Methods: We searched EBSCOhost, Scopus, Web of Science, and PubMed databases until 1 September 2020 to find studies that used HMD systems for assessing or training balance. The methodological quality was assessed using a modified version of Downs and Black. We also appraised the risk of bias using Risk of Bias Assessment tool for Non-randomized Studies (RoBANS). Results: A total of 19 articles (637 participants) were included for review. Despite heterogenous age ranges and clinical conditions across studies, VR HMD systems were valid to assess balance and could be useful for fall prevention and for improving postural control and gait patterns. These systems also have the capacity to differentiate healthy and balance-impaired individuals. During VR versions of traditional balance tests, older adults generally acquire a cautious behavior and take more time to complete the tasks. Conclusion: VR HMD systems can offer ecologically valid scenarios to assess and train functional balance and can be used alone or in addition to other interventions. New norms and protocols should be defined according to participants' age, health status, and severity of their illness when using VR HMD systems for balance assessment and training. For safe and feasible training, attention must be given to display type, VR elements and scenarios, duration of exposure, and system usability. Due to high risk of bias and overall poor quality of the studies, further research is needed on the effectiveness of HMD VR training in older adults.

Virtual Reality Applications for Neurological Disease: A Review.

Recent advancements in Virtual Reality (VR) immersive technologies provide new tools for the development of novel and promising applications for neurological rehabilitation. The purpose of this paper is to review the emerging VR applications developed for the evaluation and treatment of patients with neurological diseases. We start by discussing the impact of novel VR tasks that encourage and facilitate the patient's empowerment and involvement in the rehabilitation process. Then, a systematic review was carried out on six well-known electronic libraries using the terms: "Virtual Reality AND Neurorehabilitation," or "Head Mounted Display AND Neurorehabilitation." This review focused on fully-immersive VR systems for which 12 relevant studies published in the time span of the last five years (from 2014 to 2019) were identified. Overall, this review paper examined the use of VR in certain neurological conditions such as dementia, stroke, spinal cord injury, Parkinson's, and multiple sclerosis. Most of the studies reveal positive results suggesting that VR is a feasible and effective tool in the treatment of neurological disorders. In addition, the finding of this systematic literature review suggested that low-cost, immersive VR technologies can prove to be effective for clinical rehabilitation in healthcare, and home-based setting with practical implications and uses. The development of VR technologies in recent years has resulted in more accessible and affordable solutions that can still provide promising results. Concluding, VR and interactive devices resulted in the development of holistic, portable, accessible, and usable systems for certain neurological disease interventions. It is expected that emerging VR technologies and tools will further facilitate the development of state of the art applications in the future, exerting a significant impact on the wellbeing of the patient.

Patient's Self-monitoring of Transurethral Surgical Images Using a Head-mounted Display.

We present an application of head-mounted display (HMD) to patient's self-monitoring of transurethral resection of bladder tumor (TURB). Six patients wore the HMD as an imaging monitor to view the operation in real-time during their TURB. Following the operation, the patients completed a questionnaire that evaluates understanding of the state of their disease and satisfaction with the HMD. As a result, monitoring the operation in real time through the HMD helped to increase patients' understanding of the state of their disease and satisfaction. For selected patients, the use of HMD could help to increase the patient's understanding of their disease.